Jul 03 20:35:08.907188 ip-10-0-131-199 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 03 20:35:08.907197 ip-10-0-131-199 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 03 20:35:08.907205 ip-10-0-131-199 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 03 20:35:08.907471 ip-10-0-131-199 systemd[1]: Failed to start Kubernetes Kubelet. Jul 03 20:35:19.447574 ip-10-0-131-199 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 03 20:35:19.447589 ip-10-0-131-199 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9046e0d9bdce436692e23495ee4b1966 -- Jul 03 20:37:08.383022 ip-10-0-131-199 systemd[1]: Starting Kubernetes Kubelet... Jul 03 20:37:08.851699 ip-10-0-131-199 systemd[1]: Started Kubernetes Kubelet. Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.779199 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784319 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784331 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784335 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784344 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784349 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784353 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:37:08.858227 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784357 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784362 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784367 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784371 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784376 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784380 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784383 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784387 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784390 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784395 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784399 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784402 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784413 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784417 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784421 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784425 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784429 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784432 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784437 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784441 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:37:08.860431 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784445 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784451 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784456 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784460 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784463 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784467 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784471 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784475 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784478 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784483 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784487 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784490 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784494 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784498 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784502 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784505 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784509 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784513 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784517 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784520 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:37:08.863018 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784524 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784528 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784538 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784542 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784546 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784551 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784557 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784561 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784565 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784569 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784573 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784577 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784581 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784584 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784588 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784592 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784596 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784599 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784603 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:37:08.864085 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784607 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784610 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784614 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784620 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784624 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784627 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784632 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784636 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784639 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784643 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784647 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784651 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784655 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784658 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784662 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784665 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784671 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784675 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784678 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784682 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:37:08.865285 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784686 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784690 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784694 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784698 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784701 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784705 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784708 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784712 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784716 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784720 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784723 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784727 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784731 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.784734 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785667 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785676 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785679 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785684 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785689 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785693 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:37:08.865931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785697 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785701 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785707 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785712 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785716 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785721 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785724 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785728 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785732 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785738 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785743 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785747 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785752 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785755 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785781 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785786 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785790 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785794 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785799 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:37:08.866698 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785803 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785808 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785811 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785816 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785820 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785824 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785828 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785832 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785836 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785840 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785844 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785850 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785856 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785860 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785864 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785869 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785873 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785878 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785882 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785886 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:37:08.869075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785889 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785893 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785897 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785901 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785905 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785909 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785913 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785917 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785921 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785926 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785929 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785933 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785937 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785941 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785946 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785950 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785953 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785957 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785960 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:37:08.870731 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785964 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785968 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785971 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785975 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785979 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785983 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785986 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785990 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785994 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.785997 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786001 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786005 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786009 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786013 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786016 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786020 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786025 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786029 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786032 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786036 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:37:08.873506 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786039 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786042 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786046 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786051 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786057 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786060 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786064 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786068 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786071 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786075 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786079 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786083 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786086 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786090 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786094 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786648 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786655 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786659 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786663 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786668 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:37:08.875701 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786672 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786676 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786680 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786684 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786688 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786692 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786696 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786700 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786703 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786707 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786711 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786715 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786718 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786722 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786726 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786730 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786737 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786741 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786745 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:37:08.882888 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786749 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786753 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786756 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786775 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786780 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786783 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786787 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786791 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786795 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786798 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786802 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786806 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786809 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786813 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786817 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786820 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786824 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786828 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786831 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786835 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:37:08.884104 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786839 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786843 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786846 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786852 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786856 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786860 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786864 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786867 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786871 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786880 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786884 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786888 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786892 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786896 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786900 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786903 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786907 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786910 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786914 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786918 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:37:08.891148 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786921 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786925 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786929 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786933 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786936 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786940 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786948 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786952 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786955 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786959 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786963 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786966 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786970 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786974 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786978 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786981 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786985 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786989 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786992 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.786998 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:37:08.892034 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787003 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787010 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787014 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787018 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787023 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787027 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787030 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787034 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787038 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787041 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787046 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787050 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787053 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787056 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787060 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787134 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787142 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787149 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787155 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787161 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787165 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 03 20:37:08.892931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787171 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787176 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787180 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787184 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787189 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787193 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787198 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787202 2558 flags.go:64] FLAG: --cgroup-root="" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787205 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787209 2558 flags.go:64] FLAG: --client-ca-file="" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787213 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787217 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787225 2558 flags.go:64] FLAG: --cluster-domain="" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787229 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787233 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787238 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787243 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787248 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787252 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787256 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787260 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787265 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787269 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787273 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 03 20:37:08.894020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787277 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787282 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787287 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787291 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787296 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787300 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787303 2558 flags.go:64] FLAG: --enable-server="true" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787308 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787313 2558 flags.go:64] FLAG: --event-burst="100" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787317 2558 flags.go:64] FLAG: --event-qps="50" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787321 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787325 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787328 2558 flags.go:64] FLAG: --eviction-hard="" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787333 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787337 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787341 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787346 2558 flags.go:64] FLAG: --eviction-soft="" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787349 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787354 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787358 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787365 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787369 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787373 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787377 2558 flags.go:64] FLAG: --feature-gates="" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787382 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 03 20:37:08.894935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787387 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787391 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787396 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787400 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787404 2558 flags.go:64] FLAG: --help="false" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787408 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-131-199.ec2.internal" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787412 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787416 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787421 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787425 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787430 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787434 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787438 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787442 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787446 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787450 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787454 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787458 2558 flags.go:64] FLAG: --kube-reserved="" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787462 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787466 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787470 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787474 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787478 2558 flags.go:64] FLAG: --lock-file="" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787482 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 03 20:37:08.896034 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787486 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787490 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787497 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787504 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787507 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787512 2558 flags.go:64] FLAG: --logging-format="text" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787516 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787520 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787524 2558 flags.go:64] FLAG: --manifest-url="" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787528 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787533 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787538 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787544 2558 flags.go:64] FLAG: --max-pods="110" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787548 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787552 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787555 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787560 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787564 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787568 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787572 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787580 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787585 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787589 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 03 20:37:08.897151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787593 2558 flags.go:64] FLAG: --pod-cidr="" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787597 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787604 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787607 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787612 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787617 2558 flags.go:64] FLAG: --port="10250" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787620 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787624 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ed4516f673dd7391" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787628 2558 flags.go:64] FLAG: --qos-reserved="" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787632 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787636 2558 flags.go:64] FLAG: --register-node="true" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787640 2558 flags.go:64] FLAG: --register-with-taints="" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787646 2558 flags.go:64] FLAG: --registry-burst="10" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787651 2558 flags.go:64] FLAG: --registry-qps="5" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787655 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787659 2558 flags.go:64] FLAG: --reserved-memory="" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787664 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787668 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787673 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787677 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787681 2558 flags.go:64] FLAG: --runonce="false" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787685 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787690 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787693 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787697 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 03 20:37:08.898252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787701 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787705 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787709 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787714 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787718 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787722 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787725 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787729 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787733 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787738 2558 flags.go:64] FLAG: --system-cgroups="" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787742 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787748 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787752 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787756 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787776 2558 flags.go:64] FLAG: --tls-min-version="" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787779 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787783 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787787 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787792 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787798 2558 flags.go:64] FLAG: --v="2" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787805 2558 flags.go:64] FLAG: --version="false" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787810 2558 flags.go:64] FLAG: --vmodule="" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787814 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.787819 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787968 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:37:08.899354 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787973 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787978 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787982 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787986 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787990 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787994 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.787998 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788002 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788005 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788009 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788013 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788017 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788021 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788025 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788029 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788032 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788036 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788040 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788044 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788048 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:37:08.900457 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788051 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788055 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788059 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788062 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788066 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788071 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788076 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788082 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788085 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788089 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788093 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788097 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788100 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788104 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788108 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788112 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788115 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788120 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788124 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788127 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:37:08.901337 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788131 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788136 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788141 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788146 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788150 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788153 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788157 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788161 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788165 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788169 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788173 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788176 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788180 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788184 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788187 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788191 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788195 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788199 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788204 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:37:08.902235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788209 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788213 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788218 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788222 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788225 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788229 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788232 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788236 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788243 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788248 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788253 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788257 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788261 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788265 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788269 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788273 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788276 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788280 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788284 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:37:08.903109 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788287 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788291 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788295 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788299 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788303 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788307 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788311 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788315 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788319 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788323 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788327 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788331 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788336 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788341 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788345 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788348 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788352 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788356 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788359 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:37:08.904000 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.788363 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.789009 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.795685 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.795701 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.797214 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.797225 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797302 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797307 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797310 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797313 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797318 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797322 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797325 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797328 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:37:08.905084 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797332 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797336 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797339 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797342 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797345 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797349 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797351 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797354 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797357 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797359 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797362 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797364 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797367 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797370 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797372 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797375 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797378 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797380 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797383 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:37:08.905753 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797386 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797391 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797403 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797406 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797409 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797413 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797415 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797418 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797420 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797423 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797425 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797428 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797431 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797433 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797436 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797438 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797441 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797443 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797446 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797449 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797451 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:37:08.906639 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797454 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797456 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797459 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797462 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797464 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797467 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797469 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797472 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797475 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797477 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797480 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797483 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797486 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797489 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797497 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797500 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797504 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797506 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797509 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797511 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:37:08.907518 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797514 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797516 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797519 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797522 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797524 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797527 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797529 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797532 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797534 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797537 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797540 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797544 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797548 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797551 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797554 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797556 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797559 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797561 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797564 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:37:08.908387 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797566 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797569 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797571 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797574 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797576 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797580 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797582 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797585 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797593 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797597 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797599 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797602 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.797607 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797746 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797751 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 20:37:08.909277 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797754 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797757 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797771 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797774 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797777 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797780 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797782 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797785 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797787 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797790 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797793 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797796 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797798 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797801 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797803 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797806 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797808 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797811 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797813 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797816 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 20:37:09.057834 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797819 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797822 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797827 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797830 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797834 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797837 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797846 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797851 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797853 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797856 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797859 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797862 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797864 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797867 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797869 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797872 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797875 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797877 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797880 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 20:37:09.073546 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797882 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797885 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797887 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797890 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797892 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797896 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797899 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797902 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797905 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797908 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797910 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797913 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797915 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797918 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797921 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797924 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797927 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797930 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797932 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 20:37:09.080989 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797935 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797945 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797948 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797951 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797953 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797956 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797958 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797961 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797963 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797965 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797968 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797971 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797973 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797976 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797978 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797981 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797984 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797986 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797989 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 20:37:09.082078 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797991 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797994 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797996 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.797999 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798001 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798004 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798007 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798009 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798012 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798016 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798018 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798021 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798023 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798026 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798029 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798037 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798040 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798043 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798045 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 20:37:09.082957 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:08.798048 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.798052 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.798165 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.800216 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.801499 2558 server.go:1019] "Starting client certificate rotation" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.801612 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.802461 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.804248 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.805729 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.817372 2558 log.go:25] "Validated CRI v1 runtime API" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.819416 2558 log.go:25] "Validated CRI v1 image API" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.820300 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 03 20:37:09.084041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.823480 2558 fs.go:135] Filesystem UUIDs: map[5e904527-c346-4f11-946f-e30a1c841fe7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7cb1fef4-bf26-4fe7-9867-f27c9cc1cf5c:/dev/nvme0n1p3] Jul 03 20:37:09.084704 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.823496 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.829495 2558 manager.go:216] Machine: {Timestamp:2026-07-03 20:37:08.828290089 +0000 UTC m=+0.325857740 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099392 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ab59b9578c8facb85f4e96c780ee0 SystemUUID:ec2ab59b-9578-c8fa-cb85-f4e96c780ee0 BootID:9046e0d9-bdce-4366-92e2-3495ee4b1966 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b4:01:06:ff:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b4:01:06:ff:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:2f:af:4a:91:ec Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.829596 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.829667 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.832501 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.832546 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-199.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.833349 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.833360 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 03 20:37:09.084933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.833370 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.833999 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.835222 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.837552 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.838629 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.838736 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.838776 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.838788 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.838806 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.838814 2558 kubelet.go:398] "Adding apiserver pod source" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.838825 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.839152 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.839167 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.841281 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.843143 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.843171 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844589 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844611 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844622 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844630 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844643 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844651 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844665 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844675 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844687 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844699 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844718 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.844730 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.845560 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.845571 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.850703 2558 server.go:1268] "Started kubelet" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.850891 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.850994 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.851152 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-199.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.851384 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.852006 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-199.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.852011 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.852594 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.855980 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 03 20:37:09.085371 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.857029 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.858095 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-199.ec2.internal.18bee2088d9d7210 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-199.ec2.internal,UID:ip-10-0-131-199.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-199.ec2.internal,},FirstTimestamp:2026-07-03 20:37:08.848816656 +0000 UTC m=+0.346384307,LastTimestamp:2026-07-03 20:37:08.848816656 +0000 UTC m=+0.346384307,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-199.ec2.internal,}" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.861132 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j9zs7" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.861430 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.861490 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.861559 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862015 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862030 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862127 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862198 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862205 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862511 2558 factory.go:153] Registering CRI-O factory Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862527 2558 factory.go:223] Registration of the crio container factory successfully Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862584 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862593 2558 factory.go:55] Registering systemd factory Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862602 2558 factory.go:223] Registration of the systemd container factory successfully Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862626 2558 factory.go:103] Registering Raw factory Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.862640 2558 manager.go:1196] Started watching for new ooms in manager Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.863098 2558 manager.go:349] Starting recovery of all containers Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.867046 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j9zs7" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.867399 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-199.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.867548 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.915512 2558 manager.go:354] Recovery completed Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.920003 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.923370 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.923396 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.923406 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.923954 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.923966 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.923980 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.923958 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.924784 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:37:09.087146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.924806 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.924816 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.925218 2558 policy_none.go:49] "None policy: Start" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.925231 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.925241 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.926309 2558 policy_none.go:47] "Start" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.961717 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.962842 2558 manager.go:337] "Starting Device Plugin manager" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.962865 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.962873 2558 server.go:87] "Starting device plugin registration server" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.963065 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.963080 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.963155 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.963220 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:08.963229 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.963647 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:08.963694 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.043824 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.045013 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.045047 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.045068 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.045104 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.046881 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.063920 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.064733 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.064757 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.064783 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.064811 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.075979 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.088481 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.075995 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-199.ec2.internal\": node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.102995 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.146052 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal"] Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.146122 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.147588 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.147619 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.147630 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.149091 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.149208 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.149241 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.149977 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.150002 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.150014 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.150040 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.150057 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.150067 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.151419 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.151437 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.152107 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientMemory" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.152123 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.152136 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeHasSufficientPID" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.162927 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1474223590267aed250fed358f37a12c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-199.ec2.internal\" (UID: \"1474223590267aed250fed358f37a12c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.187430 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-199.ec2.internal\" not found" node="ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.190710 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-199.ec2.internal\" not found" node="ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.224820 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.203129 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.263022 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a1f7cc60cbfc2673fe126a4c141bfe0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal\" (UID: \"5a1f7cc60cbfc2673fe126a4c141bfe0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.263042 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a1f7cc60cbfc2673fe126a4c141bfe0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal\" (UID: \"5a1f7cc60cbfc2673fe126a4c141bfe0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.263099 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1474223590267aed250fed358f37a12c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-199.ec2.internal\" (UID: \"1474223590267aed250fed358f37a12c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.263141 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1474223590267aed250fed358f37a12c-config\") pod \"kube-apiserver-proxy-ip-10-0-131-199.ec2.internal\" (UID: \"1474223590267aed250fed358f37a12c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.304886 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.363545 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a1f7cc60cbfc2673fe126a4c141bfe0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal\" (UID: \"5a1f7cc60cbfc2673fe126a4c141bfe0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.363607 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5a1f7cc60cbfc2673fe126a4c141bfe0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal\" (UID: \"5a1f7cc60cbfc2673fe126a4c141bfe0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.363653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a1f7cc60cbfc2673fe126a4c141bfe0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal\" (UID: \"5a1f7cc60cbfc2673fe126a4c141bfe0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.394303 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.363701 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a1f7cc60cbfc2673fe126a4c141bfe0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal\" (UID: \"5a1f7cc60cbfc2673fe126a4c141bfe0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.566353 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.407436 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.566353 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.492140 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.566353 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.494465 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.566353 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.507982 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.745796 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.609160 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-199.ec2.internal\" not found" Jul 03 20:37:09.745796 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.678739 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.762534 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.785032 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.786878 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.800626 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.800768 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.800822 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.800841 2558 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://abe8000b7f49f41fabf3f96dbf53ae7b-7e98f44b5bddae44.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.131.199:57418->34.234.182.187:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.840871 2558 apiserver.go:52] "Watching apiserver" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.861890 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.869475 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 20:32:09 +0000 UTC" deadline="2028-02-06 09:30:32.655797124 +0000 UTC" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.869549 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13980h53m22.786255083s" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.871356 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.872535 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-kwnjh","openshift-network-operator/iptables-alerter-vrxkz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9","openshift-dns/node-resolver-zb5jf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal","openshift-multus/multus-vv8r5","openshift-multus/network-metrics-daemon-85jm6","openshift-network-diagnostics/network-check-target-xxg6k","openshift-ovn-kubernetes/ovnkube-node-xqhz6","kube-system/global-pull-secret-syncer-fcjwp","kube-system/konnectivity-agent-84zjd","kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wcqqt","openshift-image-registry/node-ca-nl4pk"] Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.874252 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.874339 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.875728 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.876853 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.877819 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.879891 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.880507 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.882117 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vv8r5" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.883581 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:09.895520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.884101 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.884188 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.884198 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.884188 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tqqw7\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.884302 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.884334 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.884831 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885211 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885246 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-78s8j\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885271 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885316 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-89j8v\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885318 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885473 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885533 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.885639 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7nf6t\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.886353 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.887640 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.887690 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.888272 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.888275 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.888297 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.889075 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.889197 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.889960 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.889980 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pj9df\"" Jul 03 20:37:09.898465 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.890774 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.912858 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:09.912933 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913139 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913244 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913286 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913378 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913395 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tlrfb\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913818 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bdxth\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913836 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.913985 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rplcp\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.914072 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.914144 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.914233 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.914357 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.914413 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.914531 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.914632 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mw6zl\"" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.928613 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g722p" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.938519 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g722p" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.962963 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969472 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-socket-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969494 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b74f1cc5-2656-4f91-b223-c5064e0611ff-cni-binary-copy\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969512 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.044722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969528 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-node-log\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969545 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysctl-conf\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969571 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-tuned\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969600 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-socket-dir-parent\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-k8s-cni-cncf-io\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969664 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca8829fc-8728-407f-a79e-7404e63396a7-iptables-alerter-script\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969683 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-tmp-dir\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969712 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969731 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zs6q\" (UniqueName: \"kubernetes.io/projected/4ad4d2cd-2960-4557-b2ea-b071c89b303e-kube-api-access-4zs6q\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969746 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-multus-certs\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969761 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-kubelet-config\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969792 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w497t\" (UniqueName: \"kubernetes.io/projected/e3dec259-8bc3-4826-96aa-924c10061798-kube-api-access-w497t\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969807 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-env-overrides\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.046863 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969833 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-lib-modules\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969849 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-etc-kubernetes\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969865 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-system-cni-dir\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969910 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0583e0b-4471-4710-ba0f-5c59e4173743-serviceca\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969949 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-netns\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969973 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-hosts-file\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.969998 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9hhd\" (UniqueName: \"kubernetes.io/projected/d0583e0b-4471-4710-ba0f-5c59e4173743-kube-api-access-f9hhd\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970024 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-etc-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-ovn\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970059 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-cni-bin\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970076 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysctl-d\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970103 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94d00516-d1ab-4de4-84d5-3d11f0cb8375-tmp\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970118 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-os-release\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970142 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-log-socket\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.047815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970179 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-modprobe-d\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970215 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-daemon-config\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970239 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970275 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970310 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dw9q\" (UniqueName: \"kubernetes.io/projected/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-kube-api-access-2dw9q\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970340 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970368 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-device-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970391 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-system-cni-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970414 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr5bq\" (UniqueName: \"kubernetes.io/projected/4d81ec3a-96eb-4c37-b1b1-c909945109ca-kube-api-access-sr5bq\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970438 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-kubelet\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970462 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-systemd-units\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970489 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-systemd\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-sys\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.049469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970539 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29xv8\" (UniqueName: \"kubernetes.io/projected/ca8829fc-8728-407f-a79e-7404e63396a7-kube-api-access-29xv8\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970570 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970595 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970619 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovnkube-config\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970643 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-cnibin\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-cni-multus\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970700 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cnibin\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970724 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-os-release\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970748 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cni-binary-copy\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970789 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970843 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/efc54219-f35d-4cbf-add6-4d42d71a6660-konnectivity-ca\") pod \"konnectivity-agent-84zjd\" (UID: \"efc54219-f35d-4cbf-add6-4d42d71a6660\") " pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970870 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-run-ovn-kubernetes\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970898 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-var-lib-kubelet\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970923 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-dbus\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.050437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970947 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0583e0b-4471-4710-ba0f-5c59e4173743-host\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970971 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-slash\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.970994 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysconfig\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971018 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-host\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971044 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-registration-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971065 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k55zk\" (UniqueName: \"kubernetes.io/projected/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-kube-api-access-k55zk\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-systemd\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971098 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-run\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971112 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/efc54219-f35d-4cbf-add6-4d42d71a6660-agent-certs\") pod \"konnectivity-agent-84zjd\" (UID: \"efc54219-f35d-4cbf-add6-4d42d71a6660\") " pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971138 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-hostroot\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971153 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971168 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-sys-fs\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-run-netns\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971197 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-var-lib-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.051895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971234 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-cni-netd\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovnkube-script-lib\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971280 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-kubernetes\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971317 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-cni-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-cni-bin\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971369 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovn-node-metrics-cert\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6twf\" (UniqueName: \"kubernetes.io/projected/94d00516-d1ab-4de4-84d5-3d11f0cb8375-kube-api-access-r6twf\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-kubelet\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971442 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-conf-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971468 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm8hm\" (UniqueName: \"kubernetes.io/projected/b74f1cc5-2656-4f91-b223-c5064e0611ff-kube-api-access-wm8hm\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca8829fc-8728-407f-a79e-7404e63396a7-host-slash\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.052883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:09.971515 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.062656 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.061879 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072176 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-node-log\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysctl-conf\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072237 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-tuned\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072257 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-socket-dir-parent\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072280 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-k8s-cni-cncf-io\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072283 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-node-log\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072304 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca8829fc-8728-407f-a79e-7404e63396a7-iptables-alerter-script\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072330 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-tmp-dir\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072381 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zs6q\" (UniqueName: \"kubernetes.io/projected/4ad4d2cd-2960-4557-b2ea-b071c89b303e-kube-api-access-4zs6q\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072397 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysctl-conf\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072415 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-k8s-cni-cncf-io\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072414 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-socket-dir-parent\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.180129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072408 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-multus-certs\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072438 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072445 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-multus-certs\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072562 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-kubelet-config\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072629 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w497t\" (UniqueName: \"kubernetes.io/projected/e3dec259-8bc3-4826-96aa-924c10061798-kube-api-access-w497t\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072691 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-env-overrides\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072719 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-lib-modules\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072651 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-kubelet-config\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072742 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-etc-kubernetes\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072823 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-tmp-dir\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072839 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-etc-kubernetes\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072865 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-system-cni-dir\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072892 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0583e0b-4471-4710-ba0f-5c59e4173743-serviceca\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072912 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-netns\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.183850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072929 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-hosts-file\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072945 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9hhd\" (UniqueName: \"kubernetes.io/projected/d0583e0b-4471-4710-ba0f-5c59e4173743-kube-api-access-f9hhd\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072970 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-etc-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072990 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-run-netns\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072993 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-ovn\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073038 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-cni-bin\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073047 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-etc-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073063 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysctl-d\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94d00516-d1ab-4de4-84d5-3d11f0cb8375-tmp\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073092 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-ovn\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073104 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-os-release\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073132 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-cni-bin\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073143 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-log-socket\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-modprobe-d\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073187 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-daemon-config\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.184857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073201 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca8829fc-8728-407f-a79e-7404e63396a7-iptables-alerter-script\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073290 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-log-socket\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073312 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysctl-d\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073338 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-os-release\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073337 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-env-overrides\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.072992 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-system-cni-dir\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073390 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073416 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dw9q\" (UniqueName: \"kubernetes.io/projected/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-kube-api-access-2dw9q\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073431 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-modprobe-d\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073431 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d0583e0b-4471-4710-ba0f-5c59e4173743-serviceca\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073442 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-device-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073496 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.185782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073511 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-lib-modules\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073516 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-system-cni-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073559 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-device-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073563 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr5bq\" (UniqueName: \"kubernetes.io/projected/4d81ec3a-96eb-4c37-b1b1-c909945109ca-kube-api-access-sr5bq\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073628 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-system-cni-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-kubelet\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073689 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-systemd-units\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073696 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-daemon-config\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073716 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-systemd\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073740 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-sys\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073767 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29xv8\" (UniqueName: \"kubernetes.io/projected/ca8829fc-8728-407f-a79e-7404e63396a7-kube-api-access-29xv8\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.073788 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073795 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073806 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-systemd-units\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.187172 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073821 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.073887 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret podName:fa3ffbd6-36ec-4ae6-8b13-09ffec020c70 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:10.573848457 +0000 UTC m=+2.071416112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret") pod "global-pull-secret-syncer-fcjwp" (UID: "fa3ffbd6-36ec-4ae6-8b13-09ffec020c70") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073910 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-sys\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073921 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovnkube-config\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073948 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-cnibin\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073965 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-systemd\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073993 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-cni-multus\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073459 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-hosts-file\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074020 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-cnibin\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074022 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cnibin\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.073948 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-etc-selinux\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074042 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074537 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-kubelet\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074753 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cnibin\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.188115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074108 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-cni-multus\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074906 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-os-release\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.074965 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cni-binary-copy\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075025 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075055 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/efc54219-f35d-4cbf-add6-4d42d71a6660-konnectivity-ca\") pod \"konnectivity-agent-84zjd\" (UID: \"efc54219-f35d-4cbf-add6-4d42d71a6660\") " pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075092 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-run-ovn-kubernetes\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075131 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-var-lib-kubelet\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075134 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-os-release\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075321 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-run-ovn-kubernetes\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075361 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-dbus\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075415 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0583e0b-4471-4710-ba0f-5c59e4173743-host\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.075428 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075443 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-slash\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075517 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-slash\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.189272 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.075518 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:10.575462395 +0000 UTC m=+2.073030052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075579 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysconfig\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075599 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-var-lib-kubelet\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075618 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-host\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075660 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-registration-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075670 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-host\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075690 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k55zk\" (UniqueName: \"kubernetes.io/projected/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-kube-api-access-k55zk\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075725 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-systemd\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075737 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-sysconfig\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075772 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-run\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075828 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/efc54219-f35d-4cbf-add6-4d42d71a6660-agent-certs\") pod \"konnectivity-agent-84zjd\" (UID: \"efc54219-f35d-4cbf-add6-4d42d71a6660\") " pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075872 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-dbus\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075880 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-hostroot\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075911 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075937 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d0583e0b-4471-4710-ba0f-5c59e4173743-host\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.190215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075959 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-sys-fs\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075971 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/efc54219-f35d-4cbf-add6-4d42d71a6660-konnectivity-ca\") pod \"konnectivity-agent-84zjd\" (UID: \"efc54219-f35d-4cbf-add6-4d42d71a6660\") " pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075998 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-run\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.075992 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-run-netns\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076056 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-registration-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076069 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-run-netns\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076078 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-cni-binary-copy\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076100 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-var-lib-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076126 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-systemd\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076056 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-var-lib-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076178 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-sys-fs\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076201 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-cni-netd\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076232 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovnkube-script-lib\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076264 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-kubernetes\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076588 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovnkube-config\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.191151 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076961 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94d00516-d1ab-4de4-84d5-3d11f0cb8375-tmp\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.076988 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-tuned\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078000 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-host-cni-netd\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078056 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-cni-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078094 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-cni-bin\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovn-node-metrics-cert\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078178 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6twf\" (UniqueName: \"kubernetes.io/projected/94d00516-d1ab-4de4-84d5-3d11f0cb8375-kube-api-access-r6twf\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078211 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-kubelet\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078241 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-conf-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm8hm\" (UniqueName: \"kubernetes.io/projected/b74f1cc5-2656-4f91-b223-c5064e0611ff-kube-api-access-wm8hm\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078296 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca8829fc-8728-407f-a79e-7404e63396a7-host-slash\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078343 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078375 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-socket-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078407 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b74f1cc5-2656-4f91-b223-c5064e0611ff-cni-binary-copy\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078437 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.192108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078480 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-kubelet\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078529 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ad4d2cd-2960-4557-b2ea-b071c89b303e-run-openvswitch\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078559 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-hostroot\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078685 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-cni-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078710 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4d81ec3a-96eb-4c37-b1b1-c909945109ca-socket-dir\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078739 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-host-var-lib-cni-bin\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.078584 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b74f1cc5-2656-4f91-b223-c5064e0611ff-multus-conf-dir\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.079189 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.079476 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/94d00516-d1ab-4de4-84d5-3d11f0cb8375-etc-kubernetes\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.079616 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca8829fc-8728-407f-a79e-7404e63396a7-host-slash\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.079723 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/efc54219-f35d-4cbf-add6-4d42d71a6660-agent-certs\") pod \"konnectivity-agent-84zjd\" (UID: \"efc54219-f35d-4cbf-add6-4d42d71a6660\") " pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.079922 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovnkube-script-lib\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.080062 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.080136 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b74f1cc5-2656-4f91-b223-c5064e0611ff-cni-binary-copy\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.081530 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ad4d2cd-2960-4557-b2ea-b071c89b303e-ovn-node-metrics-cert\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.193271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.087988 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29xv8\" (UniqueName: \"kubernetes.io/projected/ca8829fc-8728-407f-a79e-7404e63396a7-kube-api-access-29xv8\") pod \"iptables-alerter-vrxkz\" (UID: \"ca8829fc-8728-407f-a79e-7404e63396a7\") " pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.088150 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9hhd\" (UniqueName: \"kubernetes.io/projected/d0583e0b-4471-4710-ba0f-5c59e4173743-kube-api-access-f9hhd\") pod \"node-ca-nl4pk\" (UID: \"d0583e0b-4471-4710-ba0f-5c59e4173743\") " pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.088219 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k55zk\" (UniqueName: \"kubernetes.io/projected/e99d1d0f-5a9b-4dea-af3d-463f3e45d615-kube-api-access-k55zk\") pod \"node-resolver-zb5jf\" (UID: \"e99d1d0f-5a9b-4dea-af3d-463f3e45d615\") " pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.088305 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.088322 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.088340 2558 projected.go:196] Error preparing data for projected volume kube-api-access-sk6rp for pod openshift-network-diagnostics/network-check-target-xxg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.088391 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp podName:87717b37-557a-42f2-a33f-d2e98bbeac72 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:10.588375731 +0000 UTC m=+2.085943399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sk6rp" (UniqueName: "kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp") pod "network-check-target-xxg6k" (UID: "87717b37-557a-42f2-a33f-d2e98bbeac72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.090455 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm8hm\" (UniqueName: \"kubernetes.io/projected/b74f1cc5-2656-4f91-b223-c5064e0611ff-kube-api-access-wm8hm\") pod \"multus-vv8r5\" (UID: \"b74f1cc5-2656-4f91-b223-c5064e0611ff\") " pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.090532 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6twf\" (UniqueName: \"kubernetes.io/projected/94d00516-d1ab-4de4-84d5-3d11f0cb8375-kube-api-access-r6twf\") pod \"tuned-wcqqt\" (UID: \"94d00516-d1ab-4de4-84d5-3d11f0cb8375\") " pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.091000 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr5bq\" (UniqueName: \"kubernetes.io/projected/4d81ec3a-96eb-4c37-b1b1-c909945109ca-kube-api-access-sr5bq\") pod \"aws-ebs-csi-driver-node-cnzx9\" (UID: \"4d81ec3a-96eb-4c37-b1b1-c909945109ca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.091018 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w497t\" (UniqueName: \"kubernetes.io/projected/e3dec259-8bc3-4826-96aa-924c10061798-kube-api-access-w497t\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.096862 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vrxkz" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.098013 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zs6q\" (UniqueName: \"kubernetes.io/projected/4ad4d2cd-2960-4557-b2ea-b071c89b303e-kube-api-access-4zs6q\") pod \"ovnkube-node-xqhz6\" (UID: \"4ad4d2cd-2960-4557-b2ea-b071c89b303e\") " pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.098282 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dw9q\" (UniqueName: \"kubernetes.io/projected/7ea07ed9-94b3-45d7-a9b7-f0cb9955f423-kube-api-access-2dw9q\") pod \"multus-additional-cni-plugins-kwnjh\" (UID: \"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423\") " pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.113311 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" Jul 03 20:37:10.194443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.122125 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.215393 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.245146 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefc54219_f35d_4cbf_add6_4d42d71a6660.slice/crio-3a9a0432c7f64ed4ffd0c32788c2b4ee2940cf5b95b7e1cf9b14663d8dc5ce18 WatchSource:0}: Error finding container 3a9a0432c7f64ed4ffd0c32788c2b4ee2940cf5b95b7e1cf9b14663d8dc5ce18: Status 404 returned error can't find the container with id 3a9a0432c7f64ed4ffd0c32788c2b4ee2940cf5b95b7e1cf9b14663d8dc5ce18 Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.250685 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.250988 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1474223590267aed250fed358f37a12c.slice/crio-6a2c291573badf653e9be1bc2af5cecb819ca3d18df0a79a5b274e43e642bd38 WatchSource:0}: Error finding container 6a2c291573badf653e9be1bc2af5cecb819ca3d18df0a79a5b274e43e642bd38: Status 404 returned error can't find the container with id 6a2c291573badf653e9be1bc2af5cecb819ca3d18df0a79a5b274e43e642bd38 Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.251571 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1f7cc60cbfc2673fe126a4c141bfe0.slice/crio-4a79a4595f0fc34813de34c716466030239f46daf08f851adb0b071c4473a3e9 WatchSource:0}: Error finding container 4a79a4595f0fc34813de34c716466030239f46daf08f851adb0b071c4473a3e9: Status 404 returned error can't find the container with id 4a79a4595f0fc34813de34c716466030239f46daf08f851adb0b071c4473a3e9 Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.263057 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d81ec3a_96eb_4c37_b1b1_c909945109ca.slice/crio-0a96bfdd72d81bf12bd8f3aff03874cf32090dd553f9a9b8c33f1a8558f787cf WatchSource:0}: Error finding container 0a96bfdd72d81bf12bd8f3aff03874cf32090dd553f9a9b8c33f1a8558f787cf: Status 404 returned error can't find the container with id 0a96bfdd72d81bf12bd8f3aff03874cf32090dd553f9a9b8c33f1a8558f787cf Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.264998 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.267674 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zb5jf" Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.272063 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nl4pk" Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.283399 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode99d1d0f_5a9b_4dea_af3d_463f3e45d615.slice/crio-2a8cd07e9b4b9589c044aef87a6c4e9bc1efc37fbefbae1d7da32b2e277fa393 WatchSource:0}: Error finding container 2a8cd07e9b4b9589c044aef87a6c4e9bc1efc37fbefbae1d7da32b2e277fa393: Status 404 returned error can't find the container with id 2a8cd07e9b4b9589c044aef87a6c4e9bc1efc37fbefbae1d7da32b2e277fa393 Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.286413 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0583e0b_4471_4710_ba0f_5c59e4173743.slice/crio-efa2f0f583e9276330d559697563dde1f357c855fdfdf67984f2cd1c302dafe4 WatchSource:0}: Error finding container efa2f0f583e9276330d559697563dde1f357c855fdfdf67984f2cd1c302dafe4: Status 404 returned error can't find the container with id efa2f0f583e9276330d559697563dde1f357c855fdfdf67984f2cd1c302dafe4 Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.309452 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.316009 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vv8r5" Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.316597 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad4d2cd_2960_4557_b2ea_b071c89b303e.slice/crio-a5a6a91c52ac0a44b41645352e02c6a6b12e4ff47dd338af4e65b4ad35bd44ba WatchSource:0}: Error finding container a5a6a91c52ac0a44b41645352e02c6a6b12e4ff47dd338af4e65b4ad35bd44ba: Status 404 returned error can't find the container with id a5a6a91c52ac0a44b41645352e02c6a6b12e4ff47dd338af4e65b4ad35bd44ba Jul 03 20:37:10.338245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.336679 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" Jul 03 20:37:10.534709 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.346629 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d00516_d1ab_4de4_84d5_3d11f0cb8375.slice/crio-3918d8b2bb221a322012b7f22ce9d33f93100d805e3310f71112194d62c4ff97 WatchSource:0}: Error finding container 3918d8b2bb221a322012b7f22ce9d33f93100d805e3310f71112194d62c4ff97: Status 404 returned error can't find the container with id 3918d8b2bb221a322012b7f22ce9d33f93100d805e3310f71112194d62c4ff97 Jul 03 20:37:10.534709 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.437098 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca8829fc_8728_407f_a79e_7404e63396a7.slice/crio-f6c22b342bfb140dd9a419aea3ab639f7298bfbf7bae18a0a44cd55f82605b5e WatchSource:0}: Error finding container f6c22b342bfb140dd9a419aea3ab639f7298bfbf7bae18a0a44cd55f82605b5e: Status 404 returned error can't find the container with id f6c22b342bfb140dd9a419aea3ab639f7298bfbf7bae18a0a44cd55f82605b5e Jul 03 20:37:10.534709 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:10.523462 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea07ed9_94b3_45d7_a9b7_f0cb9955f423.slice/crio-394f1e03becb7f95fc2e7cc80ecfe613ae42f91d1a41cfd57a4a632f6ef9112e WatchSource:0}: Error finding container 394f1e03becb7f95fc2e7cc80ecfe613ae42f91d1a41cfd57a4a632f6ef9112e: Status 404 returned error can't find the container with id 394f1e03becb7f95fc2e7cc80ecfe613ae42f91d1a41cfd57a4a632f6ef9112e Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.581485 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.581598 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.581621 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.581670 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:11.581652324 +0000 UTC m=+3.079219968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.581701 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.581745 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret podName:fa3ffbd6-36ec-4ae6-8b13-09ffec020c70 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:11.581732456 +0000 UTC m=+3.079300099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret") pod "global-pull-secret-syncer-fcjwp" (UID: "fa3ffbd6-36ec-4ae6-8b13-09ffec020c70") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.682943 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.683097 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.683118 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.683129 2558 projected.go:196] Error preparing data for projected volume kube-api-access-sk6rp for pod openshift-network-diagnostics/network-check-target-xxg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:10.718319 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:10.683175 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp podName:87717b37-557a-42f2-a33f-d2e98bbeac72 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:11.683161144 +0000 UTC m=+3.180728785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk6rp" (UniqueName: "kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp") pod "network-check-target-xxg6k" (UID: "87717b37-557a-42f2-a33f-d2e98bbeac72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:10.872517 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.752279 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:37:10.941807 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.941014 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 20:32:10 +0000 UTC" deadline="2028-02-22 08:25:28.032371187 +0000 UTC" Jul 03 20:37:10.941807 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:10.941068 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14363h48m17.091307795s" Jul 03 20:37:11.050683 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.050540 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:11.050872 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.050684 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:11.050872 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.050780 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:11.051059 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.050889 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:11.054407 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.054336 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zb5jf" event={"ID":"e99d1d0f-5a9b-4dea-af3d-463f3e45d615","Type":"ContainerStarted","Data":"2a8cd07e9b4b9589c044aef87a6c4e9bc1efc37fbefbae1d7da32b2e277fa393"} Jul 03 20:37:11.061017 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.060868 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" event={"ID":"5a1f7cc60cbfc2673fe126a4c141bfe0","Type":"ContainerStarted","Data":"4a79a4595f0fc34813de34c716466030239f46daf08f851adb0b071c4473a3e9"} Jul 03 20:37:11.068242 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.068213 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vrxkz" event={"ID":"ca8829fc-8728-407f-a79e-7404e63396a7","Type":"ContainerStarted","Data":"f6c22b342bfb140dd9a419aea3ab639f7298bfbf7bae18a0a44cd55f82605b5e"} Jul 03 20:37:11.078624 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.078574 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" event={"ID":"94d00516-d1ab-4de4-84d5-3d11f0cb8375","Type":"ContainerStarted","Data":"3918d8b2bb221a322012b7f22ce9d33f93100d805e3310f71112194d62c4ff97"} Jul 03 20:37:11.094646 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.094613 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" event={"ID":"4d81ec3a-96eb-4c37-b1b1-c909945109ca","Type":"ContainerStarted","Data":"0a96bfdd72d81bf12bd8f3aff03874cf32090dd553f9a9b8c33f1a8558f787cf"} Jul 03 20:37:11.109073 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.109038 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-84zjd" event={"ID":"efc54219-f35d-4cbf-add6-4d42d71a6660","Type":"ContainerStarted","Data":"3a9a0432c7f64ed4ffd0c32788c2b4ee2940cf5b95b7e1cf9b14663d8dc5ce18"} Jul 03 20:37:11.124169 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.124051 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" event={"ID":"1474223590267aed250fed358f37a12c","Type":"ContainerStarted","Data":"6a2c291573badf653e9be1bc2af5cecb819ca3d18df0a79a5b274e43e642bd38"} Jul 03 20:37:11.149649 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.149610 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerStarted","Data":"394f1e03becb7f95fc2e7cc80ecfe613ae42f91d1a41cfd57a4a632f6ef9112e"} Jul 03 20:37:11.181565 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.181525 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vv8r5" event={"ID":"b74f1cc5-2656-4f91-b223-c5064e0611ff","Type":"ContainerStarted","Data":"e44a9f5724f9440aa9e2e3cc7a34fb857cd4c24108868687641292b9c6ebe725"} Jul 03 20:37:11.193953 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.193915 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"a5a6a91c52ac0a44b41645352e02c6a6b12e4ff47dd338af4e65b4ad35bd44ba"} Jul 03 20:37:11.199451 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.199215 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nl4pk" event={"ID":"d0583e0b-4471-4710-ba0f-5c59e4173743","Type":"ContainerStarted","Data":"efa2f0f583e9276330d559697563dde1f357c855fdfdf67984f2cd1c302dafe4"} Jul 03 20:37:11.590910 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.590818 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:11.590910 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.590894 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:11.591132 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.591033 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:11.591132 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.591097 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:13.591078923 +0000 UTC m=+5.088646565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:11.591543 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.591522 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:11.591628 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.591577 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret podName:fa3ffbd6-36ec-4ae6-8b13-09ffec020c70 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:13.591562052 +0000 UTC m=+5.089129694 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret") pod "global-pull-secret-syncer-fcjwp" (UID: "fa3ffbd6-36ec-4ae6-8b13-09ffec020c70") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:11.692246 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.691778 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:11.692246 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.691988 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:37:11.692246 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.692011 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:37:11.692246 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.692026 2558 projected.go:196] Error preparing data for projected volume kube-api-access-sk6rp for pod openshift-network-diagnostics/network-check-target-xxg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:11.692246 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:11.692083 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp podName:87717b37-557a-42f2-a33f-d2e98bbeac72 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:13.692065448 +0000 UTC m=+5.189633109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk6rp" (UniqueName: "kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp") pod "network-check-target-xxg6k" (UID: "87717b37-557a-42f2-a33f-d2e98bbeac72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:11.942020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.941928 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 20:32:10 +0000 UTC" deadline="2028-01-18 18:52:03.321630163 +0000 UTC" Jul 03 20:37:11.942020 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:11.941969 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13534h14m51.379666197s" Jul 03 20:37:12.046139 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:12.046107 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:12.046317 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:12.046250 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:12.196046 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:12.195956 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 20:37:13.048076 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:13.048049 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:13.048549 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.048180 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:13.048625 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:13.048561 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:13.048690 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.048677 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:13.612514 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:13.612466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:13.612700 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:13.612549 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:13.612700 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.612690 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:13.612836 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.612767 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:17.612747057 +0000 UTC m=+9.110314711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:13.613252 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.613228 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:13.613375 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.613290 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret podName:fa3ffbd6-36ec-4ae6-8b13-09ffec020c70 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:17.613274582 +0000 UTC m=+9.110842228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret") pod "global-pull-secret-syncer-fcjwp" (UID: "fa3ffbd6-36ec-4ae6-8b13-09ffec020c70") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:13.713246 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:13.713211 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:13.713432 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.713338 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:37:13.713432 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.713353 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:37:13.713432 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.713362 2558 projected.go:196] Error preparing data for projected volume kube-api-access-sk6rp for pod openshift-network-diagnostics/network-check-target-xxg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:13.713432 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:13.713406 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp podName:87717b37-557a-42f2-a33f-d2e98bbeac72 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:17.713393314 +0000 UTC m=+9.210960956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk6rp" (UniqueName: "kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp") pod "network-check-target-xxg6k" (UID: "87717b37-557a-42f2-a33f-d2e98bbeac72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:14.045348 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:14.045293 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:14.045550 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:14.045435 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:15.046225 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:15.046193 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:15.046225 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:15.046217 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:15.046720 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:15.046329 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:15.046720 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:15.046478 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:16.046318 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:16.045861 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:16.046318 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:16.045979 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:17.045431 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:17.045358 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:17.045613 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.045508 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:17.045947 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:17.045705 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:17.045947 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.045845 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:17.649732 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:17.649244 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:17.649732 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:17.649313 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:17.649732 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.649443 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:17.649732 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.649500 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:25.649484314 +0000 UTC m=+17.147051958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:17.650364 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.649840 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:17.650364 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.649906 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret podName:fa3ffbd6-36ec-4ae6-8b13-09ffec020c70 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:25.649887029 +0000 UTC m=+17.147454681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret") pod "global-pull-secret-syncer-fcjwp" (UID: "fa3ffbd6-36ec-4ae6-8b13-09ffec020c70") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:17.750358 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:17.750296 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:17.750506 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.750493 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:37:17.750555 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.750517 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:37:17.750555 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.750532 2558 projected.go:196] Error preparing data for projected volume kube-api-access-sk6rp for pod openshift-network-diagnostics/network-check-target-xxg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:17.750639 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:17.750602 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp podName:87717b37-557a-42f2-a33f-d2e98bbeac72 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:25.750579645 +0000 UTC m=+17.248147285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk6rp" (UniqueName: "kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp") pod "network-check-target-xxg6k" (UID: "87717b37-557a-42f2-a33f-d2e98bbeac72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:18.046269 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:18.046188 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:18.046445 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:18.046320 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:19.047209 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:19.046677 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:19.047209 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:19.046818 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:19.047209 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:19.046961 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:19.047209 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:19.047184 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:20.045897 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:20.045717 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:20.045897 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:20.045871 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:21.045714 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:21.045664 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:21.046178 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:21.045817 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:21.046178 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:21.045666 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:21.046311 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:21.046207 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:22.046243 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:22.046211 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:22.046684 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:22.046327 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:23.045637 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:23.045555 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:23.045828 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:23.045553 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:23.045828 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:23.045691 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:23.045828 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:23.045795 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:24.045993 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:24.045953 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:24.046432 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:24.046084 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:25.046245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:25.046004 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:25.046714 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:25.046042 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:25.046714 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.046353 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:25.046714 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.046452 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:25.714626 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:25.714593 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:25.714813 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:25.714653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:25.714813 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.714775 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:25.714906 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.714827 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:41.714812484 +0000 UTC m=+33.212380126 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:25.714906 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.714775 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:25.714986 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.714913 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret podName:fa3ffbd6-36ec-4ae6-8b13-09ffec020c70 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:41.714897793 +0000 UTC m=+33.212465448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret") pod "global-pull-secret-syncer-fcjwp" (UID: "fa3ffbd6-36ec-4ae6-8b13-09ffec020c70") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:25.815444 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:25.815413 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:25.815630 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.815576 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:37:25.815630 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.815599 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:37:25.815630 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.815610 2558 projected.go:196] Error preparing data for projected volume kube-api-access-sk6rp for pod openshift-network-diagnostics/network-check-target-xxg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:25.815797 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:25.815671 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp podName:87717b37-557a-42f2-a33f-d2e98bbeac72 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:41.815652492 +0000 UTC m=+33.313220144 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk6rp" (UniqueName: "kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp") pod "network-check-target-xxg6k" (UID: "87717b37-557a-42f2-a33f-d2e98bbeac72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:26.046136 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:26.046055 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:26.046290 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:26.046165 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:27.046084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:27.045967 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:27.046084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:27.045978 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:27.046314 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:27.046094 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:27.046314 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:27.046230 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:28.045576 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:28.045545 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:28.045755 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:28.045667 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:29.046498 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:29.046470 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:29.046955 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:29.046576 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:29.046955 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:29.046611 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:29.046955 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:29.046723 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:30.046195 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:30.046162 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:30.046345 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:30.046287 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:31.045521 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.045498 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:31.045893 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.045520 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:31.045893 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:31.045633 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:31.045893 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:31.045803 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:31.255142 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.255100 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vv8r5" event={"ID":"b74f1cc5-2656-4f91-b223-c5064e0611ff","Type":"ContainerStarted","Data":"b8f1ff1448dc4a958436e289f7660a13791c5a5c6f82a073ca40361a3c6c20c1"} Jul 03 20:37:31.270510 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.270476 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" event={"ID":"94d00516-d1ab-4de4-84d5-3d11f0cb8375","Type":"ContainerStarted","Data":"91cb3ef896a544db3bee5ba6a59f63e4e2c9b01ad668b54668e1bbc321064550"} Jul 03 20:37:31.277145 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.277116 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" event={"ID":"1474223590267aed250fed358f37a12c","Type":"ContainerStarted","Data":"0383f4fa220f6809914b56ee29595eb3fba78d44ed34cd0c267307837a63dc89"} Jul 03 20:37:31.285325 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.285284 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vv8r5" podStartSLOduration=1.622036679 podStartE2EDuration="22.28526524s" totalImagesPullingTime="20.663228561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:31.28500974 +0000 UTC m=+22.782577401" watchObservedRunningTime="2026-07-03 20:37:31.28526524 +0000 UTC m=+22.782832902" Jul 03 20:37:31.344526 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.344074 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-199.ec2.internal" podStartSLOduration=21.344051766 podStartE2EDuration="21.344051766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:31.31472052 +0000 UTC m=+22.812288181" watchObservedRunningTime="2026-07-03 20:37:31.344051766 +0000 UTC m=+22.841619441" Jul 03 20:37:31.344526 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:31.344373 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wcqqt" podStartSLOduration=1.74336699 podStartE2EDuration="22.34435811s" totalImagesPullingTime="20.60099112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:31.343654688 +0000 UTC m=+22.841222352" watchObservedRunningTime="2026-07-03 20:37:31.34435811 +0000 UTC m=+22.841925770" Jul 03 20:37:32.046327 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.046148 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:32.047076 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:32.046417 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:32.280526 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.280441 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-84zjd" event={"ID":"efc54219-f35d-4cbf-add6-4d42d71a6660","Type":"ContainerStarted","Data":"aecedeac253795b43b910f4f588559ddb5010b595194cea39a52e8878f7e98ef"} Jul 03 20:37:32.281979 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.281953 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ea07ed9-94b3-45d7-a9b7-f0cb9955f423" containerID="fcbcd812397a8d72c54d42f190e0580c00c9018ba6074b3aac60bc685d68ed35" exitCode=0 Jul 03 20:37:32.282073 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.282042 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerDied","Data":"fcbcd812397a8d72c54d42f190e0580c00c9018ba6074b3aac60bc685d68ed35"} Jul 03 20:37:32.284993 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.284971 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"bd1d2aeffa0dfe01c6b585b4f148e6cde3d094e04b13b6da30fe57da687ad179"} Jul 03 20:37:32.285093 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.285048 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"1c3279275ef976768bc17a4924c48573896c9df735ea0ac543a151000d886470"} Jul 03 20:37:32.285154 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.285101 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"b1e75c2346298f356708190f5cbc50987f5dbd85ae8593307292ae799db77621"} Jul 03 20:37:32.285209 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.285154 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"975b7e5baa3d4451670537542d3467f5eaa07e0b962c463daedaa7cd1ca5c02d"} Jul 03 20:37:32.285258 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.285218 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"df035f60cee1b8f192a26ac08bf09697708e6f7319d14e99a83691f391ea0043"} Jul 03 20:37:32.285311 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.285267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"d3691d5b5aafe00c1ccdc269b6a73c876ace1618529cd61eda57878aec1a7eaf"} Jul 03 20:37:32.286325 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.286302 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nl4pk" event={"ID":"d0583e0b-4471-4710-ba0f-5c59e4173743","Type":"ContainerStarted","Data":"fe466a811575e7c3674bd929191045aa4c81e88e7ac1d952f82ddc95a3b88ed6"} Jul 03 20:37:32.287662 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.287640 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zb5jf" event={"ID":"e99d1d0f-5a9b-4dea-af3d-463f3e45d615","Type":"ContainerStarted","Data":"a2a7658fb799ac06caf2c1efe0460417dab76817becec8b8198c270a5b926df9"} Jul 03 20:37:32.288970 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.288950 2558 generic.go:356] "Generic (PLEG): container finished" podID="5a1f7cc60cbfc2673fe126a4c141bfe0" containerID="a16ece84845638f87dace5ecb2c62e9e609ae62a7e55769a0b84a4ede9c68d4d" exitCode=0 Jul 03 20:37:32.289059 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.289013 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" event={"ID":"5a1f7cc60cbfc2673fe126a4c141bfe0","Type":"ContainerDied","Data":"a16ece84845638f87dace5ecb2c62e9e609ae62a7e55769a0b84a4ede9c68d4d"} Jul 03 20:37:32.290298 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.290278 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" event={"ID":"4d81ec3a-96eb-4c37-b1b1-c909945109ca","Type":"ContainerStarted","Data":"02df9be4edcfb4bf5c1dc53eb8409afa06157673b978c0f3e760c26c137ee984"} Jul 03 20:37:32.296371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.296340 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-84zjd" podStartSLOduration=2.752954372 podStartE2EDuration="23.296328766s" totalImagesPullingTime="20.543374394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:32.295883538 +0000 UTC m=+23.793451202" watchObservedRunningTime="2026-07-03 20:37:32.296328766 +0000 UTC m=+23.793896417" Jul 03 20:37:32.310696 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.310655 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zb5jf" podStartSLOduration=2.787325239 podStartE2EDuration="23.310643122s" totalImagesPullingTime="20.523317883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:32.310530503 +0000 UTC m=+23.808098164" watchObservedRunningTime="2026-07-03 20:37:32.310643122 +0000 UTC m=+23.808210773" Jul 03 20:37:32.326175 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:32.326132 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nl4pk" podStartSLOduration=2.805598681 podStartE2EDuration="23.326116124s" totalImagesPullingTime="20.520517443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:32.325933336 +0000 UTC m=+23.823501000" watchObservedRunningTime="2026-07-03 20:37:32.326116124 +0000 UTC m=+23.823683784" Jul 03 20:37:33.046008 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.045978 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:33.046208 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:33.046095 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:33.046208 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.046141 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:33.046320 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:33.046237 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:33.185028 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.185001 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 03 20:37:33.294673 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.294635 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vrxkz" event={"ID":"ca8829fc-8728-407f-a79e-7404e63396a7","Type":"ContainerStarted","Data":"551cce43de28fad0ece11f3bf1160732290d9ace2d08a36b181f767eb5e492b4"} Jul 03 20:37:33.296782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.296698 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" event={"ID":"4d81ec3a-96eb-4c37-b1b1-c909945109ca","Type":"ContainerStarted","Data":"a191a6c20af053ca3c37895c39295b3d2452361a79a6dcbf97f38f0f4f180296"} Jul 03 20:37:33.298722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.298693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" event={"ID":"5a1f7cc60cbfc2673fe126a4c141bfe0","Type":"ContainerStarted","Data":"69ffcc247b0e8c4c4c35b6284fd993f430723b27ff8c5833494a8395051087b1"} Jul 03 20:37:33.310046 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.310002 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vrxkz" podStartSLOduration=3.939901462 podStartE2EDuration="24.309990255s" totalImagesPullingTime="20.370088793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:33.309712972 +0000 UTC m=+24.807280632" watchObservedRunningTime="2026-07-03 20:37:33.309990255 +0000 UTC m=+24.807557916" Jul 03 20:37:33.324471 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.324428 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-199.ec2.internal" podStartSLOduration=23.32441583 podStartE2EDuration="23.32441583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:33.324166964 +0000 UTC m=+24.821734640" watchObservedRunningTime="2026-07-03 20:37:33.32441583 +0000 UTC m=+24.821983490" Jul 03 20:37:33.443395 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.443358 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:33.443883 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.443864 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:33.978507 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.978412 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T20:37:33.185024104Z","UUID":"44522c2f-d8a0-4b2d-b900-d03747e9b6a8","Handler":null,"Name":"","Endpoint":""} Jul 03 20:37:33.980785 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.980340 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 03 20:37:33.980785 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:33.980378 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 03 20:37:34.046253 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:34.046225 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:34.046371 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:34.046350 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:34.304743 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:34.304640 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"71bed23a0fd4795c660da402f1e46eb9ebdcdb666b1607dc5ac6ee609398635d"} Jul 03 20:37:34.305313 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:34.305291 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:34.305869 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:34.305848 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-84zjd" Jul 03 20:37:35.049177 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:35.049105 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:35.049177 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:35.049128 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:35.049388 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:35.049229 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:35.049388 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:35.049349 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:35.308486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:35.308409 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" event={"ID":"4d81ec3a-96eb-4c37-b1b1-c909945109ca","Type":"ContainerStarted","Data":"92fadffb670ab8a34474a5dfd62580ade0f38ddf492a224bb2c07482ad172650"} Jul 03 20:37:35.328288 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:35.328245 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cnzx9" podStartSLOduration=3.058150972 podStartE2EDuration="26.328231681s" totalImagesPullingTime="23.270080709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:35.327935252 +0000 UTC m=+26.825502931" watchObservedRunningTime="2026-07-03 20:37:35.328231681 +0000 UTC m=+26.825799365" Jul 03 20:37:36.046017 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:36.045975 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:36.046198 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:36.046091 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:37.046122 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:37.046040 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:37.046524 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:37.046175 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:37.046524 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:37.046226 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:37.046524 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:37.046339 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:38.045544 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.045377 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:38.045678 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:38.045616 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:38.316739 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.316662 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ea07ed9-94b3-45d7-a9b7-f0cb9955f423" containerID="b7dc1a77dea2a8bd3f3ead539cebc44417874421ce62cc46829a5b2c116681cd" exitCode=0 Jul 03 20:37:38.317562 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.316745 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerDied","Data":"b7dc1a77dea2a8bd3f3ead539cebc44417874421ce62cc46829a5b2c116681cd"} Jul 03 20:37:38.320313 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.320290 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" event={"ID":"4ad4d2cd-2960-4557-b2ea-b071c89b303e","Type":"ContainerStarted","Data":"2deb43f245065757b20614e494a1b50bc49c31f1c515834520eba82487a02a4d"} Jul 03 20:37:38.320542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.320526 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:38.320609 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.320597 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:38.335519 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.335495 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:38.365379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:38.365338 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" podStartSLOduration=8.515814384 podStartE2EDuration="29.36532607s" totalImagesPullingTime="20.849511686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:38.364012028 +0000 UTC m=+29.861579707" watchObservedRunningTime="2026-07-03 20:37:38.36532607 +0000 UTC m=+29.862893730" Jul 03 20:37:39.047115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.047078 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:39.047362 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:39.047192 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:39.047641 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.047621 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:39.047755 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:39.047735 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:39.325274 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.325179 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerStarted","Data":"7aa2ed4ecad922b75f0117326166b8482331c18aab3d946d4b41f5cc2df8f54e"} Jul 03 20:37:39.325823 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.325801 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:39.346204 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.346184 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:37:39.467076 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.467047 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fcjwp"] Jul 03 20:37:39.467219 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.467175 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:39.467271 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:39.467251 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:39.470802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.470747 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xxg6k"] Jul 03 20:37:39.470929 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.470869 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:39.470987 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:39.470964 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:39.471459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.471435 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-85jm6"] Jul 03 20:37:39.471583 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:39.471528 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:39.471644 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:39.471598 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:40.329677 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:40.329645 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ea07ed9-94b3-45d7-a9b7-f0cb9955f423" containerID="7aa2ed4ecad922b75f0117326166b8482331c18aab3d946d4b41f5cc2df8f54e" exitCode=0 Jul 03 20:37:40.330175 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:40.329731 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerDied","Data":"7aa2ed4ecad922b75f0117326166b8482331c18aab3d946d4b41f5cc2df8f54e"} Jul 03 20:37:41.046221 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:41.046191 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:41.046377 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:41.046200 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:41.046377 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.046318 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:41.046377 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:41.046204 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:41.046512 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.046395 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:41.046512 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.046455 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:41.728308 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:41.728216 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:41.728308 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:41.728268 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:41.728708 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.728358 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:41.728708 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.728379 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:41.728708 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.728412 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:38:13.728395956 +0000 UTC m=+65.225963595 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 20:37:41.728708 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.728444 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret podName:fa3ffbd6-36ec-4ae6-8b13-09ffec020c70 nodeName:}" failed. No retries permitted until 2026-07-03 20:38:13.728424873 +0000 UTC m=+65.225992513 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret") pod "global-pull-secret-syncer-fcjwp" (UID: "fa3ffbd6-36ec-4ae6-8b13-09ffec020c70") : object "kube-system"/"original-pull-secret" not registered Jul 03 20:37:41.828580 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:41.828542 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:41.828731 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.828713 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 20:37:41.828798 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.828733 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 20:37:41.828798 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.828745 2558 projected.go:196] Error preparing data for projected volume kube-api-access-sk6rp for pod openshift-network-diagnostics/network-check-target-xxg6k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:41.828878 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:41.828808 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp podName:87717b37-557a-42f2-a33f-d2e98bbeac72 nodeName:}" failed. No retries permitted until 2026-07-03 20:38:13.828794811 +0000 UTC m=+65.326362448 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sk6rp" (UniqueName: "kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp") pod "network-check-target-xxg6k" (UID: "87717b37-557a-42f2-a33f-d2e98bbeac72") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 20:37:42.336872 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:42.336836 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ea07ed9-94b3-45d7-a9b7-f0cb9955f423" containerID="5d17cc90c0719ff385cfb74e7de274b10d59f5c668e7557b05071a3f8e7119f6" exitCode=0 Jul 03 20:37:42.337017 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:42.336905 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerDied","Data":"5d17cc90c0719ff385cfb74e7de274b10d59f5c668e7557b05071a3f8e7119f6"} Jul 03 20:37:43.045885 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.045850 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:43.045885 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.045880 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:43.046436 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.045851 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:43.046436 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:43.045970 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxg6k" podUID="87717b37-557a-42f2-a33f-d2e98bbeac72" Jul 03 20:37:43.046436 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:43.046099 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fcjwp" podUID="fa3ffbd6-36ec-4ae6-8b13-09ffec020c70" Jul 03 20:37:43.046436 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:43.046208 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:37:43.866967 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.866903 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-199.ec2.internal" event="NodeReady" Jul 03 20:37:43.867104 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.867018 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 03 20:37:43.905640 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.905613 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d5859fc4f-5gb28"] Jul 03 20:37:43.907671 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.907646 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm"] Jul 03 20:37:43.907808 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.907786 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:43.909361 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.909341 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:43.910139 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.910115 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 03 20:37:43.910253 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.910235 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dxprn\"" Jul 03 20:37:43.910313 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.910247 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 03 20:37:43.910479 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.910466 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 03 20:37:43.911973 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.911950 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 03 20:37:43.912176 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.912156 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wm9df\"" Jul 03 20:37:43.912248 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.912227 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 03 20:37:43.922282 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.922066 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 03 20:37:43.922671 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.922643 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hqvwk"] Jul 03 20:37:43.924649 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.924628 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm"] Jul 03 20:37:43.924804 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.924746 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:43.926253 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.926185 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d5859fc4f-5gb28"] Jul 03 20:37:43.927709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.927682 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lzmlt\"" Jul 03 20:37:43.929203 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.928971 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 03 20:37:43.929288 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.929267 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 03 20:37:43.929437 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.929417 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 03 20:37:43.930408 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.930390 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rpcz2"] Jul 03 20:37:43.932494 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.932475 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:43.939544 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.939521 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 03 20:37:43.939637 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.939620 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v2wzp\"" Jul 03 20:37:43.940291 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.940273 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 03 20:37:43.941561 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.941543 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqvwk"] Jul 03 20:37:43.956588 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:43.956560 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rpcz2"] Jul 03 20:37:44.045838 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.045806 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2nqs\" (UniqueName: \"kubernetes.io/projected/fa57c721-8f8e-4496-afec-72a573a690b4-kube-api-access-n2nqs\") pod \"ingress-canary-hqvwk\" (UID: \"fa57c721-8f8e-4496-afec-72a573a690b4\") " pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:44.045838 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.045840 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.045865 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4798d9f3-76b8-471b-a6dc-e4882536cb4e-tmp-dir\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.045882 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2stjg\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-kube-api-access-2stjg\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.045976 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-image-registry-private-configuration\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046025 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046054 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4798d9f3-76b8-471b-a6dc-e4882536cb4e-config-volume\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046071 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-certificates\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046104 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046128 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-trusted-ca\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046143 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/377561ea-6240-45a1-820b-b0971f7f56a2-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-bound-sa-token\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046193 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qbmr\" (UniqueName: \"kubernetes.io/projected/4798d9f3-76b8-471b-a6dc-e4882536cb4e-kube-api-access-4qbmr\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046214 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d1704bd-4282-4647-99e0-498fa8c9a70e-ca-trust-extracted\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046237 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-installation-pull-secrets\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.046839 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.046264 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa57c721-8f8e-4496-afec-72a573a690b4-cert\") pod \"ingress-canary-hqvwk\" (UID: \"fa57c721-8f8e-4496-afec-72a573a690b4\") " pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:44.147249 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147165 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.147249 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147213 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-trusted-ca\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.147249 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147238 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/377561ea-6240-45a1-820b-b0971f7f56a2-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147261 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-bound-sa-token\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qbmr\" (UniqueName: \"kubernetes.io/projected/4798d9f3-76b8-471b-a6dc-e4882536cb4e-kube-api-access-4qbmr\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147389 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d1704bd-4282-4647-99e0-498fa8c9a70e-ca-trust-extracted\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147416 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-installation-pull-secrets\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147444 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa57c721-8f8e-4496-afec-72a573a690b4-cert\") pod \"ingress-canary-hqvwk\" (UID: \"fa57c721-8f8e-4496-afec-72a573a690b4\") " pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147488 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2nqs\" (UniqueName: \"kubernetes.io/projected/fa57c721-8f8e-4496-afec-72a573a690b4-kube-api-access-n2nqs\") pod \"ingress-canary-hqvwk\" (UID: \"fa57c721-8f8e-4496-afec-72a573a690b4\") " pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147510 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.147542 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147542 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4798d9f3-76b8-471b-a6dc-e4882536cb4e-tmp-dir\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.147964 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.147566 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2stjg\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-kube-api-access-2stjg\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.147964 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.147734 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:37:44.147964 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.147753 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:37:44.147964 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.147831 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:44.64781165 +0000 UTC m=+36.145379308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:37:44.148177 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148035 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4798d9f3-76b8-471b-a6dc-e4882536cb4e-tmp-dir\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.148177 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148037 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/377561ea-6240-45a1-820b-b0971f7f56a2-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:44.148309 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148263 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-trusted-ca\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.148382 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148323 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d1704bd-4282-4647-99e0-498fa8c9a70e-ca-trust-extracted\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.148382 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148320 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-image-registry-private-configuration\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.148482 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.148397 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:37:44.148482 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.148459 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:44.648444181 +0000 UTC m=+36.146011849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:37:44.148589 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148506 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:44.148589 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148551 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4798d9f3-76b8-471b-a6dc-e4882536cb4e-config-volume\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.148589 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.148574 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-certificates\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.148733 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.148656 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:37:44.148733 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.148721 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:44.648707085 +0000 UTC m=+36.146274723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:37:44.149087 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.149064 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-certificates\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.149237 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.149213 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4798d9f3-76b8-471b-a6dc-e4882536cb4e-config-volume\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.152123 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.152094 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-image-registry-private-configuration\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.152240 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.152130 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa57c721-8f8e-4496-afec-72a573a690b4-cert\") pod \"ingress-canary-hqvwk\" (UID: \"fa57c721-8f8e-4496-afec-72a573a690b4\") " pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:44.152326 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.152308 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-installation-pull-secrets\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.161804 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.161781 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-bound-sa-token\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.162167 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.162124 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qbmr\" (UniqueName: \"kubernetes.io/projected/4798d9f3-76b8-471b-a6dc-e4882536cb4e-kube-api-access-4qbmr\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.162329 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.162309 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2nqs\" (UniqueName: \"kubernetes.io/projected/fa57c721-8f8e-4496-afec-72a573a690b4-kube-api-access-n2nqs\") pod \"ingress-canary-hqvwk\" (UID: \"fa57c721-8f8e-4496-afec-72a573a690b4\") " pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:44.169929 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.169905 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2stjg\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-kube-api-access-2stjg\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.238690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.238660 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hqvwk" Jul 03 20:37:44.389167 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.389141 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hqvwk"] Jul 03 20:37:44.392383 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:37:44.392358 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa57c721_8f8e_4496_afec_72a573a690b4.slice/crio-903e3f85518f1cb0988afaedeca9e0af45758cb0a4b7f40ec89f908cadc5e6bc WatchSource:0}: Error finding container 903e3f85518f1cb0988afaedeca9e0af45758cb0a4b7f40ec89f908cadc5e6bc: Status 404 returned error can't find the container with id 903e3f85518f1cb0988afaedeca9e0af45758cb0a4b7f40ec89f908cadc5e6bc Jul 03 20:37:44.652584 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.652544 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:44.652740 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.652643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:44.652740 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.652690 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:37:44.652740 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:44.652693 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:44.652941 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.652755 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:45.652732859 +0000 UTC m=+37.150300498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:37:44.652941 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.652793 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:37:44.652941 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.652808 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:37:44.652941 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.652827 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:37:44.652941 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.652847 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:45.652831023 +0000 UTC m=+37.150398663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:37:44.652941 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:44.652873 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:45.652858617 +0000 UTC m=+37.150426270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:37:45.046325 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.046296 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:37:45.046325 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.046320 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:37:45.047115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.046296 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:37:45.052099 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.051916 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 20:37:45.052099 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.052066 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4xwrm\"" Jul 03 20:37:45.053206 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.053184 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ml64n\"" Jul 03 20:37:45.053315 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.053219 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 20:37:45.053370 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.053191 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 20:37:45.053789 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.053738 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 20:37:45.347165 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.347081 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqvwk" event={"ID":"fa57c721-8f8e-4496-afec-72a573a690b4","Type":"ContainerStarted","Data":"903e3f85518f1cb0988afaedeca9e0af45758cb0a4b7f40ec89f908cadc5e6bc"} Jul 03 20:37:45.661851 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.661748 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:45.661851 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.661847 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:45.662062 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:45.661897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:45.662062 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:45.661939 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:37:45.662062 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:45.661963 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:37:45.662062 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:45.661977 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:37:45.662062 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:45.662025 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:47.66200561 +0000 UTC m=+39.159573249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:37:45.662062 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:45.662042 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:47.662033707 +0000 UTC m=+39.159601345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:37:45.662335 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:45.662066 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:37:45.662335 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:45.662130 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:47.66211194 +0000 UTC m=+39.159679591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:37:47.679245 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:47.679207 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:47.679260 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:47.679370 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:47.679393 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:47.679409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:47.679453 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:51.679432678 +0000 UTC m=+43.177000333 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:47.679483 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:47.679490 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:47.679523 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:51.679513395 +0000 UTC m=+43.177081043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:37:47.679638 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:47.679546 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:51.679529363 +0000 UTC m=+43.177097007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:37:48.356794 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:48.356596 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ea07ed9-94b3-45d7-a9b7-f0cb9955f423" containerID="d2abe6720792a3ece79fd56ba77de20b9aca68479654f5a91c204796aedd064f" exitCode=0 Jul 03 20:37:48.356988 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:48.356676 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerDied","Data":"d2abe6720792a3ece79fd56ba77de20b9aca68479654f5a91c204796aedd064f"} Jul 03 20:37:48.358296 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:48.358274 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hqvwk" event={"ID":"fa57c721-8f8e-4496-afec-72a573a690b4","Type":"ContainerStarted","Data":"9029028c96810ebf4192717ba70f8dad6db71378f4bd1ba829b51f526925a6a5"} Jul 03 20:37:48.395524 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:48.395477 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hqvwk" podStartSLOduration=1.9908164799999999 podStartE2EDuration="5.395460633s" totalImagesPullingTime="3.404644153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:48.394725671 +0000 UTC m=+39.892293347" watchObservedRunningTime="2026-07-03 20:37:48.395460633 +0000 UTC m=+39.893028271" Jul 03 20:37:49.363960 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:49.363930 2558 generic.go:356] "Generic (PLEG): container finished" podID="7ea07ed9-94b3-45d7-a9b7-f0cb9955f423" containerID="d4b47fbc0dbcee8720ce1749faa8ecf03ffaea0ad1bee68a68e92543b6c10212" exitCode=0 Jul 03 20:37:49.364324 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:49.364019 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerDied","Data":"d4b47fbc0dbcee8720ce1749faa8ecf03ffaea0ad1bee68a68e92543b6c10212"} Jul 03 20:37:50.369295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:50.369261 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" event={"ID":"7ea07ed9-94b3-45d7-a9b7-f0cb9955f423","Type":"ContainerStarted","Data":"10e4e6052396e98def748c2e6c6142675c54e1c9ff46dbcb7df8418aad910c97"} Jul 03 20:37:50.394746 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:50.394700 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kwnjh" podStartSLOduration=8.158603462 podStartE2EDuration="41.394685582s" totalImagesPullingTime="33.23608212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:37:50.39305044 +0000 UTC m=+41.890618099" watchObservedRunningTime="2026-07-03 20:37:50.394685582 +0000 UTC m=+41.892253265" Jul 03 20:37:51.713780 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:51.713720 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:51.713791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:51.713824 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:51.713869 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:51.713890 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:51.713914 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:51.713923 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:51.713944 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:59.713930015 +0000 UTC m=+51.211497656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:51.713957 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:37:59.713951132 +0000 UTC m=+51.211518770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:37:51.714164 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:51.713967 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:37:59.713961821 +0000 UTC m=+51.211529459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:37:59.775268 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:59.775230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:59.775293 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:37:59.775326 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:59.775425 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:59.775450 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:59.775456 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:59.775478 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:59.775511 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:38:15.775490702 +0000 UTC m=+67.273058342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:59.775534 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:38:15.775522028 +0000 UTC m=+67.273089667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:37:59.775726 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:37:59.775564 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:38:15.775550536 +0000 UTC m=+67.273118177 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:38:11.342056 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:11.342030 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xqhz6" Jul 03 20:38:13.774575 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.774523 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:38:13.774988 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.774610 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:38:13.777481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.777454 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 20:38:13.777579 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.777549 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 20:38:13.785544 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:13.785521 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 03 20:38:13.785620 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:13.785589 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:39:17.785567485 +0000 UTC m=+129.283135144 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : secret "metrics-daemon-secret" not found Jul 03 20:38:13.788592 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.788572 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa3ffbd6-36ec-4ae6-8b13-09ffec020c70-original-pull-secret\") pod \"global-pull-secret-syncer-fcjwp\" (UID: \"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70\") " pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:38:13.859933 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.859903 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fcjwp" Jul 03 20:38:13.875105 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.875066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:38:13.877797 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.877783 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 20:38:13.888345 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.888325 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 20:38:13.898924 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.898906 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk6rp\" (UniqueName: \"kubernetes.io/projected/87717b37-557a-42f2-a33f-d2e98bbeac72-kube-api-access-sk6rp\") pod \"network-check-target-xxg6k\" (UID: \"87717b37-557a-42f2-a33f-d2e98bbeac72\") " pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:38:13.976214 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:13.976187 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fcjwp"] Jul 03 20:38:13.978882 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:38:13.978859 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3ffbd6_36ec_4ae6_8b13_09ffec020c70.slice/crio-804f94bc612b911189e5c193b1a4dc517c7f8581d1d98345d2a410f0c245bf2e WatchSource:0}: Error finding container 804f94bc612b911189e5c193b1a4dc517c7f8581d1d98345d2a410f0c245bf2e: Status 404 returned error can't find the container with id 804f94bc612b911189e5c193b1a4dc517c7f8581d1d98345d2a410f0c245bf2e Jul 03 20:38:14.174300 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:14.174271 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ml64n\"" Jul 03 20:38:14.182311 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:14.182292 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:38:14.298345 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:14.298246 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xxg6k"] Jul 03 20:38:14.300475 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:38:14.300449 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87717b37_557a_42f2_a33f_d2e98bbeac72.slice/crio-32ae22c899af0927378e556f513a8ffa9d7e898ffe1c37f0b7b051993ab4c195 WatchSource:0}: Error finding container 32ae22c899af0927378e556f513a8ffa9d7e898ffe1c37f0b7b051993ab4c195: Status 404 returned error can't find the container with id 32ae22c899af0927378e556f513a8ffa9d7e898ffe1c37f0b7b051993ab4c195 Jul 03 20:38:14.427320 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:14.427238 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xxg6k" event={"ID":"87717b37-557a-42f2-a33f-d2e98bbeac72","Type":"ContainerStarted","Data":"32ae22c899af0927378e556f513a8ffa9d7e898ffe1c37f0b7b051993ab4c195"} Jul 03 20:38:14.428355 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:14.428327 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fcjwp" event={"ID":"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70","Type":"ContainerStarted","Data":"804f94bc612b911189e5c193b1a4dc517c7f8581d1d98345d2a410f0c245bf2e"} Jul 03 20:38:15.791345 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:15.791306 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:15.791364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:15.791410 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:15.791448 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:15.791471 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:15.791529 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:15.791536 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:15.791550 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:38:47.791517848 +0000 UTC m=+99.289085510 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:15.791593 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:38:47.791580442 +0000 UTC m=+99.289148088 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:38:15.791819 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:15.791606 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:38:47.791598085 +0000 UTC m=+99.289165727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:38:19.442397 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:19.442354 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fcjwp" event={"ID":"fa3ffbd6-36ec-4ae6-8b13-09ffec020c70","Type":"ContainerStarted","Data":"e02cd8098801dda3cf7dd95b7ca9a40aa859fe56d7f2f9f73197db0bc056ef32"} Jul 03 20:38:19.443701 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:19.443673 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xxg6k" event={"ID":"87717b37-557a-42f2-a33f-d2e98bbeac72","Type":"ContainerStarted","Data":"c393ec2453c7ebe8c4cd1c76395d0f66d9ea5063f37ff9dcffcd4f1d0bc92d31"} Jul 03 20:38:19.443831 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:19.443814 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:38:19.458131 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:19.458092 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fcjwp" podStartSLOduration=65.398495718 podStartE2EDuration="1m10.458081626s" totalImagesPullingTime="5.059585908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:19.457895357 +0000 UTC m=+70.955463033" watchObservedRunningTime="2026-07-03 20:38:19.458081626 +0000 UTC m=+70.955649286" Jul 03 20:38:19.472886 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:19.472854 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xxg6k" podStartSLOduration=65.738688333 podStartE2EDuration="1m10.472841342s" totalImagesPullingTime="4.734153009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:38:19.472281221 +0000 UTC m=+70.969848886" watchObservedRunningTime="2026-07-03 20:38:19.472841342 +0000 UTC m=+70.970408998" Jul 03 20:38:47.819040 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:47.819006 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:47.819054 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:47.819097 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:47.819174 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:47.819191 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:47.819204 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:47.819228 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:47.819264 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:39:51.819241612 +0000 UTC m=+163.316809272 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:47.819279 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:39:51.819272804 +0000 UTC m=+163.316840442 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:38:47.819547 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:38:47.819289 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:39:51.819283885 +0000 UTC m=+163.316851523 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:38:50.448448 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:38:50.448421 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xxg6k" Jul 03 20:39:17.829420 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:39:17.829377 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:39:17.829930 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:17.829500 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 03 20:39:17.829930 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:17.829566 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:19.829551323 +0000 UTC m=+251.327118961 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : secret "metrics-daemon-secret" not found Jul 03 20:39:46.920483 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:46.920438 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" Jul 03 20:39:46.929773 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:46.929738 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" podUID="377561ea-6240-45a1-820b-b0971f7f56a2" Jul 03 20:39:46.945890 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:46.945867 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rpcz2" podUID="4798d9f3-76b8-471b-a6dc-e4882536cb4e" Jul 03 20:39:47.650589 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:39:47.650561 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rpcz2" Jul 03 20:39:47.650792 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:39:47.650563 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:39:47.650792 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:39:47.650563 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:39:48.066812 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:48.066759 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:39:51.859389 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:39:51.859341 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:39:51.859406 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:39:51.859458 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:51.859496 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:51.859555 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:51.859500 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:51.859558 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert podName:377561ea-6240-45a1-820b-b0971f7f56a2 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:53.859542058 +0000 UTC m=+285.357109701 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xz8vm" (UID: "377561ea-6240-45a1-820b-b0971f7f56a2") : secret "networking-console-plugin-cert" not found Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:51.859598 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d5859fc4f-5gb28: secret "image-registry-tls" not found Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:51.859613 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls podName:4798d9f3-76b8-471b-a6dc-e4882536cb4e nodeName:}" failed. No retries permitted until 2026-07-03 20:41:53.859602212 +0000 UTC m=+285.357169851 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls") pod "dns-default-rpcz2" (UID: "4798d9f3-76b8-471b-a6dc-e4882536cb4e") : secret "dns-default-metrics-tls" not found Jul 03 20:39:51.859788 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:39:51.859638 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls podName:0d1704bd-4282-4647-99e0-498fa8c9a70e nodeName:}" failed. No retries permitted until 2026-07-03 20:41:53.859626565 +0000 UTC m=+285.357194207 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls") pod "image-registry-d5859fc4f-5gb28" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e") : secret "image-registry-tls" not found Jul 03 20:40:03.046258 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:40:03.046170 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:41:16.446712 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.446652 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8"] Jul 03 20:41:16.448422 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.448406 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95"] Jul 03 20:41:16.448538 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.448520 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.450124 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.450105 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w"] Jul 03 20:41:16.450258 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.450239 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:16.451439 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.451414 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 03 20:41:16.451562 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.451416 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 03 20:41:16.451696 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.451682 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:41:16.451785 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.451712 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 03 20:41:16.451854 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.451802 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vs8sq\"" Jul 03 20:41:16.452173 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.452153 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5dbt7"] Jul 03 20:41:16.452289 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.452276 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.452555 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.452449 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 03 20:41:16.452555 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.452463 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2f5kz\"" Jul 03 20:41:16.452849 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.452834 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:41:16.452906 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.452838 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 03 20:41:16.453820 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.453804 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.454969 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.454949 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 03 20:41:16.455067 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.454973 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 03 20:41:16.455223 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.455207 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 03 20:41:16.456444 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.456429 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 03 20:41:16.456533 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.456433 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 03 20:41:16.456533 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.456507 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 03 20:41:16.456533 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.456433 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dgsm4\"" Jul 03 20:41:16.456652 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.456512 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 03 20:41:16.456776 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.456750 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 03 20:41:16.457447 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.457429 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-44l6l\"" Jul 03 20:41:16.462519 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.462499 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 03 20:41:16.464018 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.463996 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8"] Jul 03 20:41:16.464735 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.464718 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95"] Jul 03 20:41:16.467025 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.467007 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w"] Jul 03 20:41:16.468121 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.468103 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5dbt7"] Jul 03 20:41:16.543559 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543525 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6638ddc3-acba-40eb-b82a-3ffeb3799103-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.543559 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6638ddc3-acba-40eb-b82a-3ffeb3799103-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.543802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543584 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scq9v\" (UniqueName: \"kubernetes.io/projected/6638ddc3-acba-40eb-b82a-3ffeb3799103-kube-api-access-scq9v\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.543802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543631 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0b6df9-93cb-4d46-b0a1-c7318a298782-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.543802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543683 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.543802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543736 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:16.543802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543783 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0b6df9-93cb-4d46-b0a1-c7318a298782-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.543802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543803 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhsf4\" (UniqueName: \"kubernetes.io/projected/0e0b6df9-93cb-4d46-b0a1-c7318a298782-kube-api-access-zhsf4\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.544021 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543824 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b97sx\" (UniqueName: \"kubernetes.io/projected/8abe3320-96c3-4dea-9607-20cebe590e8d-kube-api-access-b97sx\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:16.544021 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543846 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e0b6df9-93cb-4d46-b0a1-c7318a298782-tmp\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.544021 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543871 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e0b6df9-93cb-4d46-b0a1-c7318a298782-serving-cert\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.544021 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543904 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.544021 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543924 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6qzp\" (UniqueName: \"kubernetes.io/projected/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-kube-api-access-b6qzp\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.544021 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.543948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0e0b6df9-93cb-4d46-b0a1-c7318a298782-snapshots\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.546611 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.546590 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88"] Jul 03 20:41:16.548507 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.548494 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" Jul 03 20:41:16.551411 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.551395 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-zjtfg\"" Jul 03 20:41:16.562534 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.562515 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88"] Jul 03 20:41:16.644285 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644252 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e0b6df9-93cb-4d46-b0a1-c7318a298782-tmp\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.644285 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644285 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e0b6df9-93cb-4d46-b0a1-c7318a298782-serving-cert\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.644443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644321 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.644443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644355 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6qzp\" (UniqueName: \"kubernetes.io/projected/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-kube-api-access-b6qzp\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.644443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0e0b6df9-93cb-4d46-b0a1-c7318a298782-snapshots\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.644553 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644526 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6638ddc3-acba-40eb-b82a-3ffeb3799103-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.644658 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644590 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6638ddc3-acba-40eb-b82a-3ffeb3799103-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.644658 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644626 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scq9v\" (UniqueName: \"kubernetes.io/projected/6638ddc3-acba-40eb-b82a-3ffeb3799103-kube-api-access-scq9v\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.644778 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644665 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e0b6df9-93cb-4d46-b0a1-c7318a298782-tmp\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.644837 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644794 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0b6df9-93cb-4d46-b0a1-c7318a298782-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.644837 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.644948 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644878 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:16.644948 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644902 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0b6df9-93cb-4d46-b0a1-c7318a298782-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.644948 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644925 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhsf4\" (UniqueName: \"kubernetes.io/projected/0e0b6df9-93cb-4d46-b0a1-c7318a298782-kube-api-access-zhsf4\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.645092 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b97sx\" (UniqueName: \"kubernetes.io/projected/8abe3320-96c3-4dea-9607-20cebe590e8d-kube-api-access-b97sx\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:16.645092 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.644987 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqnlm\" (UniqueName: \"kubernetes.io/projected/a89424ca-2f2e-4204-a840-3db523b87579-kube-api-access-jqnlm\") pod \"network-check-source-989f8b8dc-w8d88\" (UID: \"a89424ca-2f2e-4204-a840-3db523b87579\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" Jul 03 20:41:16.645092 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:16.645031 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 20:41:16.645092 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.645070 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0e0b6df9-93cb-4d46-b0a1-c7318a298782-snapshots\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.645275 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:16.645102 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls podName:8abe3320-96c3-4dea-9607-20cebe590e8d nodeName:}" failed. No retries permitted until 2026-07-03 20:41:17.145083169 +0000 UTC m=+248.642650815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ckr95" (UID: "8abe3320-96c3-4dea-9607-20cebe590e8d") : secret "samples-operator-tls" not found Jul 03 20:41:16.645275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.645238 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.645392 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.645308 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0b6df9-93cb-4d46-b0a1-c7318a298782-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.645392 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.645339 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6638ddc3-acba-40eb-b82a-3ffeb3799103-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.645392 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:16.645356 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:16.645518 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:16.645413 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls podName:c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe nodeName:}" failed. No retries permitted until 2026-07-03 20:41:17.14540033 +0000 UTC m=+248.642967971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hx74w" (UID: "c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe") : secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:16.645898 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.645881 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e0b6df9-93cb-4d46-b0a1-c7318a298782-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.646995 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.646969 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e0b6df9-93cb-4d46-b0a1-c7318a298782-serving-cert\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.647343 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.647322 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6638ddc3-acba-40eb-b82a-3ffeb3799103-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.652157 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.652138 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6qzp\" (UniqueName: \"kubernetes.io/projected/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-kube-api-access-b6qzp\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:16.653171 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.653151 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhsf4\" (UniqueName: \"kubernetes.io/projected/0e0b6df9-93cb-4d46-b0a1-c7318a298782-kube-api-access-zhsf4\") pod \"insights-operator-5c847dcfc4-5dbt7\" (UID: \"0e0b6df9-93cb-4d46-b0a1-c7318a298782\") " pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.653522 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.653505 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scq9v\" (UniqueName: \"kubernetes.io/projected/6638ddc3-acba-40eb-b82a-3ffeb3799103-kube-api-access-scq9v\") pod \"kube-storage-version-migrator-operator-846bf668f7-vdzf8\" (UID: \"6638ddc3-acba-40eb-b82a-3ffeb3799103\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.653601 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.653588 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b97sx\" (UniqueName: \"kubernetes.io/projected/8abe3320-96c3-4dea-9607-20cebe590e8d-kube-api-access-b97sx\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:16.746117 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.746067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqnlm\" (UniqueName: \"kubernetes.io/projected/a89424ca-2f2e-4204-a840-3db523b87579-kube-api-access-jqnlm\") pod \"network-check-source-989f8b8dc-w8d88\" (UID: \"a89424ca-2f2e-4204-a840-3db523b87579\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" Jul 03 20:41:16.753007 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.752989 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqnlm\" (UniqueName: \"kubernetes.io/projected/a89424ca-2f2e-4204-a840-3db523b87579-kube-api-access-jqnlm\") pod \"network-check-source-989f8b8dc-w8d88\" (UID: \"a89424ca-2f2e-4204-a840-3db523b87579\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" Jul 03 20:41:16.759734 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.759716 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" Jul 03 20:41:16.779401 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.779383 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" Jul 03 20:41:16.857235 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.856817 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" Jul 03 20:41:16.890901 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.890856 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8"] Jul 03 20:41:16.894466 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:16.894440 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6638ddc3_acba_40eb_b82a_3ffeb3799103.slice/crio-acdf934c172573e2d03e4a340ad693cdce4595a55c1be82ab4049b4bb748d3f1 WatchSource:0}: Error finding container acdf934c172573e2d03e4a340ad693cdce4595a55c1be82ab4049b4bb748d3f1: Status 404 returned error can't find the container with id acdf934c172573e2d03e4a340ad693cdce4595a55c1be82ab4049b4bb748d3f1 Jul 03 20:41:16.907340 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.907295 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-5dbt7"] Jul 03 20:41:16.909790 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:16.909752 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0b6df9_93cb_4d46_b0a1_c7318a298782.slice/crio-35672b0587fdb3a1306c228d03eca194257f7fd73b11b6614fe3669e3badcd47 WatchSource:0}: Error finding container 35672b0587fdb3a1306c228d03eca194257f7fd73b11b6614fe3669e3badcd47: Status 404 returned error can't find the container with id 35672b0587fdb3a1306c228d03eca194257f7fd73b11b6614fe3669e3badcd47 Jul 03 20:41:16.976043 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:16.976016 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88"] Jul 03 20:41:16.978714 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:16.978690 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89424ca_2f2e_4204_a840_3db523b87579.slice/crio-ad320640b4c96a84b4a944878c955d3942176fee981ed25746cfeb5fab65a770 WatchSource:0}: Error finding container ad320640b4c96a84b4a944878c955d3942176fee981ed25746cfeb5fab65a770: Status 404 returned error can't find the container with id ad320640b4c96a84b4a944878c955d3942176fee981ed25746cfeb5fab65a770 Jul 03 20:41:17.149265 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:17.149236 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:17.149397 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:17.149278 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:17.149397 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:17.149389 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:17.149480 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:17.149428 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 20:41:17.149480 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:17.149464 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls podName:c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe nodeName:}" failed. No retries permitted until 2026-07-03 20:41:18.149444069 +0000 UTC m=+249.647011724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hx74w" (UID: "c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe") : secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:17.149480 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:17.149481 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls podName:8abe3320-96c3-4dea-9607-20cebe590e8d nodeName:}" failed. No retries permitted until 2026-07-03 20:41:18.149473248 +0000 UTC m=+249.647040886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ckr95" (UID: "8abe3320-96c3-4dea-9607-20cebe590e8d") : secret "samples-operator-tls" not found Jul 03 20:41:17.857758 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:17.857707 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" event={"ID":"6638ddc3-acba-40eb-b82a-3ffeb3799103","Type":"ContainerStarted","Data":"acdf934c172573e2d03e4a340ad693cdce4595a55c1be82ab4049b4bb748d3f1"} Jul 03 20:41:17.859787 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:17.859694 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" event={"ID":"a89424ca-2f2e-4204-a840-3db523b87579","Type":"ContainerStarted","Data":"74ca547da6a48599748ec35b5f1e545be66056e4ed19880019dd07b0621fc686"} Jul 03 20:41:17.859787 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:17.859743 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" event={"ID":"a89424ca-2f2e-4204-a840-3db523b87579","Type":"ContainerStarted","Data":"ad320640b4c96a84b4a944878c955d3942176fee981ed25746cfeb5fab65a770"} Jul 03 20:41:17.861359 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:17.861330 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" event={"ID":"0e0b6df9-93cb-4d46-b0a1-c7318a298782","Type":"ContainerStarted","Data":"35672b0587fdb3a1306c228d03eca194257f7fd73b11b6614fe3669e3badcd47"} Jul 03 20:41:17.875457 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:17.875409 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-w8d88" podStartSLOduration=1.875393257 podStartE2EDuration="1.875393257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:17.874190262 +0000 UTC m=+249.371757923" watchObservedRunningTime="2026-07-03 20:41:17.875393257 +0000 UTC m=+249.372960909" Jul 03 20:41:18.157077 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:18.156990 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:18.157077 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:18.157047 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:18.157317 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:18.157149 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 20:41:18.157317 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:18.157151 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:18.157317 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:18.157211 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls podName:8abe3320-96c3-4dea-9607-20cebe590e8d nodeName:}" failed. No retries permitted until 2026-07-03 20:41:20.157192759 +0000 UTC m=+251.654760397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ckr95" (UID: "8abe3320-96c3-4dea-9607-20cebe590e8d") : secret "samples-operator-tls" not found Jul 03 20:41:18.157317 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:18.157225 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls podName:c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe nodeName:}" failed. No retries permitted until 2026-07-03 20:41:20.157218822 +0000 UTC m=+251.654786460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hx74w" (UID: "c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe") : secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:19.869773 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:19.869740 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:41:19.870201 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:19.869828 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" event={"ID":"0e0b6df9-93cb-4d46-b0a1-c7318a298782","Type":"ContainerStarted","Data":"d0e241c4c6af1be13d2d03003349a684e4cad180d0b4e417a00b0db7b4250142"} Jul 03 20:41:19.870201 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:19.869907 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 03 20:41:19.870201 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:19.869973 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs podName:e3dec259-8bc3-4826-96aa-924c10061798 nodeName:}" failed. No retries permitted until 2026-07-03 20:43:21.869954329 +0000 UTC m=+373.367521975 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs") pod "network-metrics-daemon-85jm6" (UID: "e3dec259-8bc3-4826-96aa-924c10061798") : secret "metrics-daemon-secret" not found Jul 03 20:41:19.871591 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:19.871571 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" event={"ID":"6638ddc3-acba-40eb-b82a-3ffeb3799103","Type":"ContainerStarted","Data":"050986c0f7e67fdc67d96ad08531bdbe4c38b5568e5982126681fc774dabdd71"} Jul 03 20:41:19.884895 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:19.884853 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" podStartSLOduration=1.509182147 podStartE2EDuration="3.884835578s" totalImagesPullingTime="2.375653431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:19.88408962 +0000 UTC m=+251.381657280" watchObservedRunningTime="2026-07-03 20:41:19.884835578 +0000 UTC m=+251.382403239" Jul 03 20:41:19.897637 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:19.897593 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" podStartSLOduration=1.505543826 podStartE2EDuration="3.897574586s" totalImagesPullingTime="2.39203076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:19.897162756 +0000 UTC m=+251.394730408" watchObservedRunningTime="2026-07-03 20:41:19.897574586 +0000 UTC m=+251.395142268" Jul 03 20:41:20.172886 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.172802 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:20.172886 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.172861 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:20.173102 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.172955 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:20.173102 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.173020 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls podName:c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe nodeName:}" failed. No retries permitted until 2026-07-03 20:41:24.173003463 +0000 UTC m=+255.670571106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hx74w" (UID: "c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe") : secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:20.173102 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.173043 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 20:41:20.173226 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.173115 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls podName:8abe3320-96c3-4dea-9607-20cebe590e8d nodeName:}" failed. No retries permitted until 2026-07-03 20:41:24.173096665 +0000 UTC m=+255.670664326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ckr95" (UID: "8abe3320-96c3-4dea-9607-20cebe590e8d") : secret "samples-operator-tls" not found Jul 03 20:41:20.609648 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.609599 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8mqll"] Jul 03 20:41:20.612115 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.612091 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.621041 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.621018 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 03 20:41:20.621681 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.621660 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 03 20:41:20.623897 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.623879 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gz87q\"" Jul 03 20:41:20.643339 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.643304 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8mqll"] Jul 03 20:41:20.677130 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.677104 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.677281 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.677260 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7c1946a-264d-4401-b552-12914faa5d86-crio-socket\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.677323 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.677294 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.677323 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.677312 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c1946a-264d-4401-b552-12914faa5d86-data-volume\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.677385 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.677345 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h267c\" (UniqueName: \"kubernetes.io/projected/c7c1946a-264d-4401-b552-12914faa5d86-kube-api-access-h267c\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.778195 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.778162 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.778195 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.778203 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c1946a-264d-4401-b552-12914faa5d86-data-volume\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.778363 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.778222 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h267c\" (UniqueName: \"kubernetes.io/projected/c7c1946a-264d-4401-b552-12914faa5d86-kube-api-access-h267c\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.778363 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.778240 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.778363 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.778299 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:41:20.778363 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.778329 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:41:20.778501 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.778376 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:21.278357662 +0000 UTC m=+252.775925300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : secret "insights-runtime-extractor-tls" not found Jul 03 20:41:20.778501 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:20.778404 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:21.278396288 +0000 UTC m=+252.775963926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : configmap "kube-rbac-proxy" not found Jul 03 20:41:20.778501 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.778480 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7c1946a-264d-4401-b552-12914faa5d86-crio-socket\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.778605 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.778548 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c7c1946a-264d-4401-b552-12914faa5d86-crio-socket\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.778635 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.778599 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c7c1946a-264d-4401-b552-12914faa5d86-data-volume\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:20.786358 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:20.786327 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h267c\" (UniqueName: \"kubernetes.io/projected/c7c1946a-264d-4401-b552-12914faa5d86-kube-api-access-h267c\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:21.281831 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.281787 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:21.282228 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.281844 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:21.282228 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:21.281913 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:41:21.282228 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:21.281950 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:41:21.282228 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:21.281993 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:22.281973161 +0000 UTC m=+253.779540864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : configmap "kube-rbac-proxy" not found Jul 03 20:41:21.282228 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:21.282009 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:22.28200226 +0000 UTC m=+253.779569898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : secret "insights-runtime-extractor-tls" not found Jul 03 20:41:21.404433 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.404398 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576"] Jul 03 20:41:21.406923 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.406907 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" Jul 03 20:41:21.409284 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.409264 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 03 20:41:21.409714 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.409657 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 03 20:41:21.410367 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.410352 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mx86g\"" Jul 03 20:41:21.416682 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.416661 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576"] Jul 03 20:41:21.483638 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.483616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdvc2\" (UniqueName: \"kubernetes.io/projected/00fe4a14-4e5c-4b68-bc91-9720552b5a1c-kube-api-access-hdvc2\") pod \"migrator-d57d58cc9-sg576\" (UID: \"00fe4a14-4e5c-4b68-bc91-9720552b5a1c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" Jul 03 20:41:21.584296 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.584267 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdvc2\" (UniqueName: \"kubernetes.io/projected/00fe4a14-4e5c-4b68-bc91-9720552b5a1c-kube-api-access-hdvc2\") pod \"migrator-d57d58cc9-sg576\" (UID: \"00fe4a14-4e5c-4b68-bc91-9720552b5a1c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" Jul 03 20:41:21.592188 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.592157 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdvc2\" (UniqueName: \"kubernetes.io/projected/00fe4a14-4e5c-4b68-bc91-9720552b5a1c-kube-api-access-hdvc2\") pod \"migrator-d57d58cc9-sg576\" (UID: \"00fe4a14-4e5c-4b68-bc91-9720552b5a1c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" Jul 03 20:41:21.717335 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.717292 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" Jul 03 20:41:21.836536 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.836465 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576"] Jul 03 20:41:21.839486 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:21.839459 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00fe4a14_4e5c_4b68_bc91_9720552b5a1c.slice/crio-d20cd6bbc203906292ca2750c2922aaa8a3174027682b92f8296956271b6e125 WatchSource:0}: Error finding container d20cd6bbc203906292ca2750c2922aaa8a3174027682b92f8296956271b6e125: Status 404 returned error can't find the container with id d20cd6bbc203906292ca2750c2922aaa8a3174027682b92f8296956271b6e125 Jul 03 20:41:21.878263 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:21.878235 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" event={"ID":"00fe4a14-4e5c-4b68-bc91-9720552b5a1c","Type":"ContainerStarted","Data":"d20cd6bbc203906292ca2750c2922aaa8a3174027682b92f8296956271b6e125"} Jul 03 20:41:22.290869 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:22.290841 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:22.291282 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:22.290888 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:22.291282 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:22.290964 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:41:22.291282 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:22.291013 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:41:22.291282 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:22.291041 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:24.291025661 +0000 UTC m=+255.788593309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : configmap "kube-rbac-proxy" not found Jul 03 20:41:22.291282 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:22.291065 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:24.291049135 +0000 UTC m=+255.788616774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : secret "insights-runtime-extractor-tls" not found Jul 03 20:41:22.596630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:22.596561 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zb5jf_e99d1d0f-5a9b-4dea-af3d-463f3e45d615/dns-node-resolver/0.log" Jul 03 20:41:23.597024 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:23.596998 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nl4pk_d0583e0b-4471-4710-ba0f-5c59e4173743/node-ca/0.log" Jul 03 20:41:23.889412 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:23.889331 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" event={"ID":"00fe4a14-4e5c-4b68-bc91-9720552b5a1c","Type":"ContainerStarted","Data":"8cc0b15d13eeac3054a76d49208ef0055f7afe93efbb6b2af934c3241c3a7a7f"} Jul 03 20:41:23.889412 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:23.889406 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" event={"ID":"00fe4a14-4e5c-4b68-bc91-9720552b5a1c","Type":"ContainerStarted","Data":"a87e30dbf8df955655a367e4ffd16332a4722bc06566c7cf67d7d6cfa1f2f60d"} Jul 03 20:41:23.903792 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:23.903734 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-sg576" podStartSLOduration=1.700790636 podStartE2EDuration="2.903720336s" totalImagesPullingTime="1.2029297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:23.902403913 +0000 UTC m=+255.399971574" watchObservedRunningTime="2026-07-03 20:41:23.903720336 +0000 UTC m=+255.401287997" Jul 03 20:41:24.063384 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.063348 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-mrskt"] Jul 03 20:41:24.066367 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.066353 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.068754 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.068730 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 03 20:41:24.068908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.068830 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7wh4v\"" Jul 03 20:41:24.068908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.068887 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 03 20:41:24.069035 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.068993 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 03 20:41:24.069102 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.069085 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 03 20:41:24.073731 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.073708 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-mrskt"] Jul 03 20:41:24.106674 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.106649 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5925c090-cc18-4ce6-a44f-05c386d9c94e-signing-key\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.106802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.106756 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5925c090-cc18-4ce6-a44f-05c386d9c94e-signing-cabundle\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.106915 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.106892 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4xlv\" (UniqueName: \"kubernetes.io/projected/5925c090-cc18-4ce6-a44f-05c386d9c94e-kube-api-access-t4xlv\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.208102 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.208045 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5925c090-cc18-4ce6-a44f-05c386d9c94e-signing-cabundle\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.208201 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.208101 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4xlv\" (UniqueName: \"kubernetes.io/projected/5925c090-cc18-4ce6-a44f-05c386d9c94e-kube-api-access-t4xlv\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.208201 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.208144 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:24.208201 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.208166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5925c090-cc18-4ce6-a44f-05c386d9c94e-signing-key\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.208308 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.208204 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:24.208308 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.208267 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:24.208376 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.208315 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 03 20:41:24.208376 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.208330 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls podName:c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe nodeName:}" failed. No retries permitted until 2026-07-03 20:41:32.208313544 +0000 UTC m=+263.705881189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hx74w" (UID: "c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe") : secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:24.208376 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.208359 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls podName:8abe3320-96c3-4dea-9607-20cebe590e8d nodeName:}" failed. No retries permitted until 2026-07-03 20:41:32.208345099 +0000 UTC m=+263.705912742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ckr95" (UID: "8abe3320-96c3-4dea-9607-20cebe590e8d") : secret "samples-operator-tls" not found Jul 03 20:41:24.208810 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.208791 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5925c090-cc18-4ce6-a44f-05c386d9c94e-signing-cabundle\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.210520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.210500 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5925c090-cc18-4ce6-a44f-05c386d9c94e-signing-key\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.215632 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.215614 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4xlv\" (UniqueName: \"kubernetes.io/projected/5925c090-cc18-4ce6-a44f-05c386d9c94e-kube-api-access-t4xlv\") pod \"service-ca-758f599c5-mrskt\" (UID: \"5925c090-cc18-4ce6-a44f-05c386d9c94e\") " pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.309375 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.309335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:24.309375 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.309384 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:24.309564 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.309455 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:41:24.309564 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.309473 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:41:24.309564 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.309518 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:28.309501746 +0000 UTC m=+259.807069384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : configmap "kube-rbac-proxy" not found Jul 03 20:41:24.309564 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:24.309535 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:28.309524182 +0000 UTC m=+259.807091821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : secret "insights-runtime-extractor-tls" not found Jul 03 20:41:24.375937 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.375902 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-mrskt" Jul 03 20:41:24.494915 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.494854 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-mrskt"] Jul 03 20:41:24.497510 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:24.497483 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5925c090_cc18_4ce6_a44f_05c386d9c94e.slice/crio-1755e1c456a9b529ec63864f7c81e96f9eef83bb90065d3c18ac74116f57614a WatchSource:0}: Error finding container 1755e1c456a9b529ec63864f7c81e96f9eef83bb90065d3c18ac74116f57614a: Status 404 returned error can't find the container with id 1755e1c456a9b529ec63864f7c81e96f9eef83bb90065d3c18ac74116f57614a Jul 03 20:41:24.597475 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.597445 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqvwk_fa57c721-8f8e-4496-afec-72a573a690b4/serve-healthcheck-canary/0.log" Jul 03 20:41:24.893527 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:24.893490 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-mrskt" event={"ID":"5925c090-cc18-4ce6-a44f-05c386d9c94e","Type":"ContainerStarted","Data":"1755e1c456a9b529ec63864f7c81e96f9eef83bb90065d3c18ac74116f57614a"} Jul 03 20:41:26.900240 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:26.900208 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-mrskt" event={"ID":"5925c090-cc18-4ce6-a44f-05c386d9c94e","Type":"ContainerStarted","Data":"625e5b7f524a33b850c7ba6e4ab3693787b8acd5dc18dcfa8266997e2e24ac14"} Jul 03 20:41:26.913935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:26.913872 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-mrskt" podStartSLOduration=1.133071067 podStartE2EDuration="2.913856704s" totalImagesPullingTime="1.780785637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:26.913755852 +0000 UTC m=+258.411323514" watchObservedRunningTime="2026-07-03 20:41:26.913856704 +0000 UTC m=+258.411424363" Jul 03 20:41:28.343563 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:28.343533 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:28.343978 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:28.343657 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:28.343978 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:28.343708 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:41:28.343978 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:28.343723 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 03 20:41:28.343978 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:28.343820 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:36.34380582 +0000 UTC m=+267.841373471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : configmap "kube-rbac-proxy" not found Jul 03 20:41:28.343978 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:28.343838 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:36.343831296 +0000 UTC m=+267.841398933 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : secret "insights-runtime-extractor-tls" not found Jul 03 20:41:32.275471 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:32.275377 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:32.275471 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:32.275466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:32.275905 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:32.275535 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:32.275905 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:32.275605 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls podName:c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe nodeName:}" failed. No retries permitted until 2026-07-03 20:41:48.27558679 +0000 UTC m=+279.773154452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-hx74w" (UID: "c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe") : secret "cluster-monitoring-operator-tls" not found Jul 03 20:41:32.277890 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:32.277869 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8abe3320-96c3-4dea-9607-20cebe590e8d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ckr95\" (UID: \"8abe3320-96c3-4dea-9607-20cebe590e8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:32.367025 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:32.366988 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" Jul 03 20:41:32.486128 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:32.486095 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95"] Jul 03 20:41:32.922706 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:32.922675 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" event={"ID":"8abe3320-96c3-4dea-9607-20cebe590e8d","Type":"ContainerStarted","Data":"a36b5f7a7e775b55b43ae768762da2f33056e66b5af41077d497b7826105247d"} Jul 03 20:41:34.930538 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:34.930503 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" event={"ID":"8abe3320-96c3-4dea-9607-20cebe590e8d","Type":"ContainerStarted","Data":"606db81affae0da7a0d8a93dd8df0cc2a78714883510ab600e8ecaace806a28f"} Jul 03 20:41:34.930939 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:34.930553 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" event={"ID":"8abe3320-96c3-4dea-9607-20cebe590e8d","Type":"ContainerStarted","Data":"df4578880787bd1fefba63fdbe87b20d2d26ddf91b4d5ae12b0c46c42e755bdf"} Jul 03 20:41:34.946012 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:34.945964 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ckr95" podStartSLOduration=17.2438686 podStartE2EDuration="18.945949448s" totalImagesPullingTime="1.702080848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:34.94460737 +0000 UTC m=+266.442175046" watchObservedRunningTime="2026-07-03 20:41:34.945949448 +0000 UTC m=+266.443517108" Jul 03 20:41:36.409799 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:36.409748 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:36.409799 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:36.409807 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:36.410325 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:36.409912 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 03 20:41:36.410325 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:36.409989 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls podName:c7c1946a-264d-4401-b552-12914faa5d86 nodeName:}" failed. No retries permitted until 2026-07-03 20:41:52.409971378 +0000 UTC m=+283.907539016 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqll" (UID: "c7c1946a-264d-4401-b552-12914faa5d86") : secret "insights-runtime-extractor-tls" not found Jul 03 20:41:36.410444 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:36.410378 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c7c1946a-264d-4401-b552-12914faa5d86-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:47.625289 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.625257 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-sd2sl"] Jul 03 20:41:47.631068 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.631051 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-sd2sl" Jul 03 20:41:47.633373 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.633343 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 03 20:41:47.633628 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.633612 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gkrq7\"" Jul 03 20:41:47.633741 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.633724 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 03 20:41:47.641081 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.641057 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-sd2sl"] Jul 03 20:41:47.698441 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.698397 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwhdb\" (UniqueName: \"kubernetes.io/projected/417b55de-5169-4826-b16a-03a27dc86a73-kube-api-access-zwhdb\") pod \"downloads-685bfb5f64-sd2sl\" (UID: \"417b55de-5169-4826-b16a-03a27dc86a73\") " pod="openshift-console/downloads-685bfb5f64-sd2sl" Jul 03 20:41:47.799379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.799345 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwhdb\" (UniqueName: \"kubernetes.io/projected/417b55de-5169-4826-b16a-03a27dc86a73-kube-api-access-zwhdb\") pod \"downloads-685bfb5f64-sd2sl\" (UID: \"417b55de-5169-4826-b16a-03a27dc86a73\") " pod="openshift-console/downloads-685bfb5f64-sd2sl" Jul 03 20:41:47.810299 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.810270 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwhdb\" (UniqueName: \"kubernetes.io/projected/417b55de-5169-4826-b16a-03a27dc86a73-kube-api-access-zwhdb\") pod \"downloads-685bfb5f64-sd2sl\" (UID: \"417b55de-5169-4826-b16a-03a27dc86a73\") " pod="openshift-console/downloads-685bfb5f64-sd2sl" Jul 03 20:41:47.940896 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:47.940824 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-sd2sl" Jul 03 20:41:48.063103 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:48.063080 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-sd2sl"] Jul 03 20:41:48.064798 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:48.064757 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417b55de_5169_4826_b16a_03a27dc86a73.slice/crio-a62b63aab0d789b205477afa44daefd5273a792a6a2bd9574ed70757763542be WatchSource:0}: Error finding container a62b63aab0d789b205477afa44daefd5273a792a6a2bd9574ed70757763542be: Status 404 returned error can't find the container with id a62b63aab0d789b205477afa44daefd5273a792a6a2bd9574ed70757763542be Jul 03 20:41:48.302774 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:48.302728 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:48.305084 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:48.305061 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-hx74w\" (UID: \"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:48.573587 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:48.573510 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" Jul 03 20:41:48.693581 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:48.693372 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w"] Jul 03 20:41:48.696038 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:48.696004 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e6301c_4b15_4210_aaf3_3bcf17c9c0fe.slice/crio-47a43c6016e0f9699e07a1296194f8a68dee813e1d309acc05eca0d125eb7f46 WatchSource:0}: Error finding container 47a43c6016e0f9699e07a1296194f8a68dee813e1d309acc05eca0d125eb7f46: Status 404 returned error can't find the container with id 47a43c6016e0f9699e07a1296194f8a68dee813e1d309acc05eca0d125eb7f46 Jul 03 20:41:48.975282 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:48.975203 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-sd2sl" event={"ID":"417b55de-5169-4826-b16a-03a27dc86a73","Type":"ContainerStarted","Data":"a62b63aab0d789b205477afa44daefd5273a792a6a2bd9574ed70757763542be"} Jul 03 20:41:48.976403 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:48.976374 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" event={"ID":"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe","Type":"ContainerStarted","Data":"47a43c6016e0f9699e07a1296194f8a68dee813e1d309acc05eca0d125eb7f46"} Jul 03 20:41:50.651502 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:50.651448 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rpcz2" podUID="4798d9f3-76b8-471b-a6dc-e4882536cb4e" Jul 03 20:41:50.651502 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:50.651473 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" podUID="377561ea-6240-45a1-820b-b0971f7f56a2" Jul 03 20:41:50.651993 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:50.651473 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" Jul 03 20:41:50.956907 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.956710 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh"] Jul 03 20:41:50.970011 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.969969 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh"] Jul 03 20:41:50.970164 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.970065 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:50.972734 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.972711 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-662z2\"" Jul 03 20:41:50.972734 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.972712 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 03 20:41:50.985689 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.985666 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" event={"ID":"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe","Type":"ContainerStarted","Data":"96e9bccc800cbdc67f8d9ddd84e4fffc19af15456e1cbb6ce269148535839795"} Jul 03 20:41:50.985834 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.985814 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rpcz2" Jul 03 20:41:50.985958 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.985818 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:41:50.985958 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:50.985927 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:41:51.001457 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:51.001416 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" podStartSLOduration=33.239799529 podStartE2EDuration="35.001403202s" totalImagesPullingTime="1.761603673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:50.999837605 +0000 UTC m=+282.497405289" watchObservedRunningTime="2026-07-03 20:41:51.001403202 +0000 UTC m=+282.498970862" Jul 03 20:41:51.125941 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:51.125893 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d226421-b1ad-4b46-9145-eb9332d73f4a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m5qjh\" (UID: \"9d226421-b1ad-4b46-9145-eb9332d73f4a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:51.227292 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:51.227213 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d226421-b1ad-4b46-9145-eb9332d73f4a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m5qjh\" (UID: \"9d226421-b1ad-4b46-9145-eb9332d73f4a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:51.227461 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:51.227342 2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 03 20:41:51.227461 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:41:51.227407 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d226421-b1ad-4b46-9145-eb9332d73f4a-tls-certificates podName:9d226421-b1ad-4b46-9145-eb9332d73f4a nodeName:}" failed. No retries permitted until 2026-07-03 20:41:51.727388265 +0000 UTC m=+283.224955904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9d226421-b1ad-4b46-9145-eb9332d73f4a-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-m5qjh" (UID: "9d226421-b1ad-4b46-9145-eb9332d73f4a") : secret "prometheus-operator-admission-webhook-tls" not found Jul 03 20:41:51.731037 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:51.731005 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d226421-b1ad-4b46-9145-eb9332d73f4a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m5qjh\" (UID: \"9d226421-b1ad-4b46-9145-eb9332d73f4a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:51.733695 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:51.733664 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9d226421-b1ad-4b46-9145-eb9332d73f4a-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-m5qjh\" (UID: \"9d226421-b1ad-4b46-9145-eb9332d73f4a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:51.881966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:51.881917 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:52.019304 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:52.019229 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh"] Jul 03 20:41:52.022374 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:52.022344 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d226421_b1ad_4b46_9145_eb9332d73f4a.slice/crio-2999eca8e3d99a8d2f6fed6a8dc3876da7e5520cd8463e5edb9a7477fafa5811 WatchSource:0}: Error finding container 2999eca8e3d99a8d2f6fed6a8dc3876da7e5520cd8463e5edb9a7477fafa5811: Status 404 returned error can't find the container with id 2999eca8e3d99a8d2f6fed6a8dc3876da7e5520cd8463e5edb9a7477fafa5811 Jul 03 20:41:52.436486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:52.436447 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:52.439089 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:52.439062 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c7c1946a-264d-4401-b552-12914faa5d86-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqll\" (UID: \"c7c1946a-264d-4401-b552-12914faa5d86\") " pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:52.720832 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:52.720739 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8mqll" Jul 03 20:41:52.872976 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:52.872668 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8mqll"] Jul 03 20:41:52.994818 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:52.994714 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" event={"ID":"9d226421-b1ad-4b46-9145-eb9332d73f4a","Type":"ContainerStarted","Data":"2999eca8e3d99a8d2f6fed6a8dc3876da7e5520cd8463e5edb9a7477fafa5811"} Jul 03 20:41:53.106830 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:53.106802 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c1946a_264d_4401_b552_12914faa5d86.slice/crio-3b9a5ff0d2bf7002bbb562238e53a3dabcf16c59a56cc2d076ef359eee21ce9f WatchSource:0}: Error finding container 3b9a5ff0d2bf7002bbb562238e53a3dabcf16c59a56cc2d076ef359eee21ce9f: Status 404 returned error can't find the container with id 3b9a5ff0d2bf7002bbb562238e53a3dabcf16c59a56cc2d076ef359eee21ce9f Jul 03 20:41:53.951080 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.951038 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:41:53.951506 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.951112 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:41:53.951506 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.951199 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:41:53.956985 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.956953 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4798d9f3-76b8-471b-a6dc-e4882536cb4e-metrics-tls\") pod \"dns-default-rpcz2\" (UID: \"4798d9f3-76b8-471b-a6dc-e4882536cb4e\") " pod="openshift-dns/dns-default-rpcz2" Jul 03 20:41:53.957580 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.957556 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"image-registry-d5859fc4f-5gb28\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:41:53.958083 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.958060 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/377561ea-6240-45a1-820b-b0971f7f56a2-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xz8vm\" (UID: \"377561ea-6240-45a1-820b-b0971f7f56a2\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:41:53.990193 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.990163 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v2wzp\"" Jul 03 20:41:53.990416 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.990398 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wm9df\"" Jul 03 20:41:53.990754 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.990601 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dxprn\"" Jul 03 20:41:53.997384 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.997139 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rpcz2" Jul 03 20:41:53.997384 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.997156 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" Jul 03 20:41:53.997384 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:53.997195 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:41:54.001639 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.001507 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqll" event={"ID":"c7c1946a-264d-4401-b552-12914faa5d86","Type":"ContainerStarted","Data":"3b9a5ff0d2bf7002bbb562238e53a3dabcf16c59a56cc2d076ef359eee21ce9f"} Jul 03 20:41:54.005269 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.004509 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" event={"ID":"9d226421-b1ad-4b46-9145-eb9332d73f4a","Type":"ContainerStarted","Data":"5018d305aedb31fecc8c0c5262136af6682013af527d9304b9b33a060d5475ca"} Jul 03 20:41:54.005269 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.005033 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:54.011593 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.011552 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" Jul 03 20:41:54.023177 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.022635 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-m5qjh" podStartSLOduration=2.886426333 podStartE2EDuration="4.022617507s" totalImagesPullingTime="1.136191174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:54.022341565 +0000 UTC m=+285.519909428" watchObservedRunningTime="2026-07-03 20:41:54.022617507 +0000 UTC m=+285.520185150" Jul 03 20:41:54.200896 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.200860 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rpcz2"] Jul 03 20:41:54.208032 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:54.207988 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4798d9f3_76b8_471b_a6dc_e4882536cb4e.slice/crio-fcd0c8d20e2240b42aae0c57f080671c9e9d12bf23df652c212512ab29bb99ce WatchSource:0}: Error finding container fcd0c8d20e2240b42aae0c57f080671c9e9d12bf23df652c212512ab29bb99ce: Status 404 returned error can't find the container with id fcd0c8d20e2240b42aae0c57f080671c9e9d12bf23df652c212512ab29bb99ce Jul 03 20:41:54.252497 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.252424 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm"] Jul 03 20:41:54.256334 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:54.256303 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod377561ea_6240_45a1_820b_b0971f7f56a2.slice/crio-f66e6c54a59f1b3f3848f67df4cea51782369ac469fd168a9766ac80cd0e7787 WatchSource:0}: Error finding container f66e6c54a59f1b3f3848f67df4cea51782369ac469fd168a9766ac80cd0e7787: Status 404 returned error can't find the container with id f66e6c54a59f1b3f3848f67df4cea51782369ac469fd168a9766ac80cd0e7787 Jul 03 20:41:54.304563 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:54.304524 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d5859fc4f-5gb28"] Jul 03 20:41:54.306746 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:41:54.306717 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1704bd_4282_4647_99e0_498fa8c9a70e.slice/crio-3f86015bc614361b2a5135cb55dc3171def76d1f061d20ca2f287c08e861cbc9 WatchSource:0}: Error finding container 3f86015bc614361b2a5135cb55dc3171def76d1f061d20ca2f287c08e861cbc9: Status 404 returned error can't find the container with id 3f86015bc614361b2a5135cb55dc3171def76d1f061d20ca2f287c08e861cbc9 Jul 03 20:41:55.014634 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:55.013706 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" event={"ID":"0d1704bd-4282-4647-99e0-498fa8c9a70e","Type":"ContainerStarted","Data":"2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0"} Jul 03 20:41:55.014634 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:55.013781 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" event={"ID":"0d1704bd-4282-4647-99e0-498fa8c9a70e","Type":"ContainerStarted","Data":"3f86015bc614361b2a5135cb55dc3171def76d1f061d20ca2f287c08e861cbc9"} Jul 03 20:41:55.014634 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:55.014565 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:41:55.017156 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:55.017089 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" event={"ID":"377561ea-6240-45a1-820b-b0971f7f56a2","Type":"ContainerStarted","Data":"f66e6c54a59f1b3f3848f67df4cea51782369ac469fd168a9766ac80cd0e7787"} Jul 03 20:41:55.019787 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:55.019707 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rpcz2" event={"ID":"4798d9f3-76b8-471b-a6dc-e4882536cb4e","Type":"ContainerStarted","Data":"fcd0c8d20e2240b42aae0c57f080671c9e9d12bf23df652c212512ab29bb99ce"} Jul 03 20:41:55.039862 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:55.039266 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" podStartSLOduration=263.039244609 podStartE2EDuration="4m23.039244609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:41:55.036982647 +0000 UTC m=+286.534550332" watchObservedRunningTime="2026-07-03 20:41:55.039244609 +0000 UTC m=+286.536812270" Jul 03 20:41:58.597797 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.597754 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86bf8ff5f7-bkcjt"] Jul 03 20:41:58.600106 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.600059 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.602592 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.602567 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 20:41:58.602592 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.602587 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 20:41:58.603903 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.603786 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 20:41:58.603903 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.603786 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 20:41:58.603903 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.603842 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 20:41:58.604194 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.604099 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7f7z7\"" Jul 03 20:41:58.609281 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.609260 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 03 20:41:58.609537 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.609509 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bf8ff5f7-bkcjt"] Jul 03 20:41:58.697155 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.697081 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-console-config\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.697155 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.697125 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-service-ca\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.697155 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.697153 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-trusted-ca-bundle\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.697383 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.697223 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-oauth-config\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.697383 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.697251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-serving-cert\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.697383 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.697303 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-oauth-serving-cert\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.697383 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.697321 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r8lx\" (UniqueName: \"kubernetes.io/projected/7c900818-d7f5-4427-bbe0-1746c8a31487-kube-api-access-4r8lx\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798040 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798013 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-oauth-serving-cert\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798040 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798043 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r8lx\" (UniqueName: \"kubernetes.io/projected/7c900818-d7f5-4427-bbe0-1746c8a31487-kube-api-access-4r8lx\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798179 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-console-config\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798249 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-service-ca\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798359 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798294 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-trusted-ca-bundle\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798359 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798350 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-oauth-config\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798380 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-serving-cert\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798934 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798888 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-oauth-serving-cert\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.798934 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.798913 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-console-config\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.799125 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.799021 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-service-ca\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.799197 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.799175 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-trusted-ca-bundle\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.800985 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.800964 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-oauth-config\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.801619 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.801598 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-serving-cert\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.805318 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.805296 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r8lx\" (UniqueName: \"kubernetes.io/projected/7c900818-d7f5-4427-bbe0-1746c8a31487-kube-api-access-4r8lx\") pod \"console-86bf8ff5f7-bkcjt\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:41:58.911868 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:41:58.911831 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:42:05.555313 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:05.555271 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bf8ff5f7-bkcjt"] Jul 03 20:42:05.560163 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:42:05.560133 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c900818_d7f5_4427_bbe0_1746c8a31487.slice/crio-1b073dadc08a35b705c53048ff21bef91cd3cb994e9fcff705b998a6df63480d WatchSource:0}: Error finding container 1b073dadc08a35b705c53048ff21bef91cd3cb994e9fcff705b998a6df63480d: Status 404 returned error can't find the container with id 1b073dadc08a35b705c53048ff21bef91cd3cb994e9fcff705b998a6df63480d Jul 03 20:42:06.047022 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:06.046961 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-85jm6" podUID="e3dec259-8bc3-4826-96aa-924c10061798" Jul 03 20:42:06.064801 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.064638 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rpcz2" event={"ID":"4798d9f3-76b8-471b-a6dc-e4882536cb4e","Type":"ContainerStarted","Data":"2bd88819334ebc719d7c1e34a5ad5253123c390f20a8bba55cc82011aaf34d98"} Jul 03 20:42:06.065495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.065230 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rpcz2" Jul 03 20:42:06.065495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.065265 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rpcz2" event={"ID":"4798d9f3-76b8-471b-a6dc-e4882536cb4e","Type":"ContainerStarted","Data":"381d2457f27f3a0af692640740e7f5cefb5efa43289ba976795cb15a40a0c689"} Jul 03 20:42:06.067022 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.066987 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bf8ff5f7-bkcjt" event={"ID":"7c900818-d7f5-4427-bbe0-1746c8a31487","Type":"ContainerStarted","Data":"1b073dadc08a35b705c53048ff21bef91cd3cb994e9fcff705b998a6df63480d"} Jul 03 20:42:06.069750 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.069679 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-sd2sl" event={"ID":"417b55de-5169-4826-b16a-03a27dc86a73","Type":"ContainerStarted","Data":"4809250d6cc568a3468110ceed7c58b6ca85c0d2bba815cc133fffe71ba677ef"} Jul 03 20:42:06.071606 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.070108 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-sd2sl" Jul 03 20:42:06.073326 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.073304 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:42:06.073429 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.073345 2558 generic.go:356] "Generic (PLEG): container finished" podID="c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe" containerID="96e9bccc800cbdc67f8d9ddd84e4fffc19af15456e1cbb6ce269148535839795" exitCode=2 Jul 03 20:42:06.073489 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.073425 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" event={"ID":"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe","Type":"ContainerDied","Data":"96e9bccc800cbdc67f8d9ddd84e4fffc19af15456e1cbb6ce269148535839795"} Jul 03 20:42:06.073785 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.073748 2558 scope.go:117] "RemoveContainer" containerID="96e9bccc800cbdc67f8d9ddd84e4fffc19af15456e1cbb6ce269148535839795" Jul 03 20:42:06.075448 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.075424 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqll" event={"ID":"c7c1946a-264d-4401-b552-12914faa5d86","Type":"ContainerStarted","Data":"7bf995392aef64a47e3824bf99a941a66006733891d1eded8632a8a5330ff524"} Jul 03 20:42:06.077585 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.077563 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" event={"ID":"377561ea-6240-45a1-820b-b0971f7f56a2","Type":"ContainerStarted","Data":"74fee36138630861bfdfe856eef47edff64ffaccfad8d410879fef023f397a27"} Jul 03 20:42:06.082421 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.082396 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-sd2sl" Jul 03 20:42:06.083061 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.082908 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rpcz2" podStartSLOduration=251.900628864 podStartE2EDuration="4m23.082891754s" totalImagesPullingTime="11.18226289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:06.080749651 +0000 UTC m=+297.578317313" watchObservedRunningTime="2026-07-03 20:42:06.082891754 +0000 UTC m=+297.580459420" Jul 03 20:42:06.114857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.114226 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-sd2sl" podStartSLOduration=1.72145243 podStartE2EDuration="19.114203834s" totalImagesPullingTime="17.392751404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:06.113342959 +0000 UTC m=+297.610910621" watchObservedRunningTime="2026-07-03 20:42:06.114203834 +0000 UTC m=+297.611771495" Jul 03 20:42:06.152638 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:06.152571 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xz8vm" podStartSLOduration=269.018305325 podStartE2EDuration="4m40.152546649s" totalImagesPullingTime="11.134241324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:06.132824934 +0000 UTC m=+297.630392596" watchObservedRunningTime="2026-07-03 20:42:06.152546649 +0000 UTC m=+297.650114311" Jul 03 20:42:07.089697 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:07.089010 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:42:07.089697 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:07.089114 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-hx74w" event={"ID":"c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe","Type":"ContainerStarted","Data":"105ac413f68cc527f0dc6df059024428033a51b089d1630c085d7bf7da4cadf7"} Jul 03 20:42:07.096745 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:07.096713 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqll" event={"ID":"c7c1946a-264d-4401-b552-12914faa5d86","Type":"ContainerStarted","Data":"444aa5a15af53e204962fb25a3703cd8d95fe22fa724f311cf7ba79f5eda01ca"} Jul 03 20:42:09.830784 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.828786 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d5859fc4f-5gb28"] Jul 03 20:42:09.880956 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.866807 2558 patch_prober.go:28] interesting pod/image-registry-d5859fc4f-5gb28 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 03 20:42:09.880956 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.866883 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 20:42:09.915755 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.913488 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-26wsk"] Jul 03 20:42:09.946790 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.945376 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:09.950347 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.949454 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 03 20:42:09.950347 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.949890 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 03 20:42:09.950347 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.950178 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-55fjf\"" Jul 03 20:42:09.950607 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.950361 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 03 20:42:09.950782 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:09.950722 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 03 20:42:10.000276 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000237 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-textfile\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000303 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000375 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-root\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sc4b\" (UniqueName: \"kubernetes.io/projected/14ee3481-359b-443f-8dad-76c0a61e57c6-kube-api-access-7sc4b\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000442 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000667 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000492 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ee3481-359b-443f-8dad-76c0a61e57c6-metrics-client-ca\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000667 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000523 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-wtmp\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000667 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000548 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.000667 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.000578 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-sys\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105055 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.104972 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ee3481-359b-443f-8dad-76c0a61e57c6-metrics-client-ca\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105055 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105026 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-wtmp\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105054 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105083 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-sys\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105123 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-textfile\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105159 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105222 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-root\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105251 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sc4b\" (UniqueName: \"kubernetes.io/projected/14ee3481-359b-443f-8dad-76c0a61e57c6-kube-api-access-7sc4b\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.105295 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.105287 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.106255 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.106206 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14ee3481-359b-443f-8dad-76c0a61e57c6-metrics-client-ca\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.106378 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.106361 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-wtmp\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.106781 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:10.106727 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 03 20:42:10.106873 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:10.106833 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls podName:14ee3481-359b-443f-8dad-76c0a61e57c6 nodeName:}" failed. No retries permitted until 2026-07-03 20:42:10.606809668 +0000 UTC m=+302.104377321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls") pod "node-exporter-26wsk" (UID: "14ee3481-359b-443f-8dad-76c0a61e57c6") : secret "node-exporter-tls" not found Jul 03 20:42:10.107268 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.107005 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-sys\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.107268 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.107249 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-textfile\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.107796 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.107748 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/14ee3481-359b-443f-8dad-76c0a61e57c6-root\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.108919 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.108237 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-accelerators-collector-config\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.112610 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.112537 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.126376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.126230 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sc4b\" (UniqueName: \"kubernetes.io/projected/14ee3481-359b-443f-8dad-76c0a61e57c6-kube-api-access-7sc4b\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.612613 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:10.612039 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:10.612613 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:10.612282 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 03 20:42:10.612613 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:10.612359 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls podName:14ee3481-359b-443f-8dad-76c0a61e57c6 nodeName:}" failed. No retries permitted until 2026-07-03 20:42:11.612340165 +0000 UTC m=+303.109907803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls") pod "node-exporter-26wsk" (UID: "14ee3481-359b-443f-8dad-76c0a61e57c6") : secret "node-exporter-tls" not found Jul 03 20:42:11.622455 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.622420 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:11.625112 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.625084 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/14ee3481-359b-443f-8dad-76c0a61e57c6-node-exporter-tls\") pod \"node-exporter-26wsk\" (UID: \"14ee3481-359b-443f-8dad-76c0a61e57c6\") " pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:11.686505 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.686470 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:42:11.686776 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.686739 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:42:11.702302 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.702284 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 03 20:42:11.762056 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.762029 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-26wsk" Jul 03 20:42:11.795087 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:42:11.795058 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14ee3481_359b_443f_8dad_76c0a61e57c6.slice/crio-7da9256ce82e636903208cefdd2951eb59c36a4b2cfda31555e1145da1299bb5 WatchSource:0}: Error finding container 7da9256ce82e636903208cefdd2951eb59c36a4b2cfda31555e1145da1299bb5: Status 404 returned error can't find the container with id 7da9256ce82e636903208cefdd2951eb59c36a4b2cfda31555e1145da1299bb5 Jul 03 20:42:11.797234 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.797211 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:42:11.803381 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.803358 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-85697c47cc-gdchw"] Jul 03 20:42:11.826974 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.826896 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85697c47cc-gdchw"] Jul 03 20:42:11.827129 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.827051 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.829923 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.829564 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-580o6ma3c9bkr\"" Jul 03 20:42:11.829923 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.829710 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tp9ck\"" Jul 03 20:42:11.829923 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.829812 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 03 20:42:11.829923 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.829898 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 03 20:42:11.830215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.829929 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 03 20:42:11.830215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.830166 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 03 20:42:11.831526 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.830352 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 03 20:42:11.926803 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.926700 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.926803 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.926744 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.927019 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.926852 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a52ff3-7341-498e-a312-aa464d65aa34-metrics-client-ca\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.927019 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.926931 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt6z6\" (UniqueName: \"kubernetes.io/projected/f8a52ff3-7341-498e-a312-aa464d65aa34-kube-api-access-zt6z6\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.927019 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.926966 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.927178 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.927028 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-grpc-tls\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.927178 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.927094 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-tls\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:11.927178 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:11.927124 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028329 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028294 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt6z6\" (UniqueName: \"kubernetes.io/projected/f8a52ff3-7341-498e-a312-aa464d65aa34-kube-api-access-zt6z6\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028513 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028346 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028513 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028375 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-grpc-tls\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028513 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028419 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-tls\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028513 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028474 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028729 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028546 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028729 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.028729 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.028645 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a52ff3-7341-498e-a312-aa464d65aa34-metrics-client-ca\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.029582 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.029555 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a52ff3-7341-498e-a312-aa464d65aa34-metrics-client-ca\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.031613 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.031579 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.032086 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.032046 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.032086 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.032071 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-tls\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.032228 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.032123 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.032533 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.032515 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-grpc-tls\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.032596 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.032541 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f8a52ff3-7341-498e-a312-aa464d65aa34-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.037914 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.037889 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt6z6\" (UniqueName: \"kubernetes.io/projected/f8a52ff3-7341-498e-a312-aa464d65aa34-kube-api-access-zt6z6\") pod \"thanos-querier-85697c47cc-gdchw\" (UID: \"f8a52ff3-7341-498e-a312-aa464d65aa34\") " pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.132175 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.132136 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26wsk" event={"ID":"14ee3481-359b-443f-8dad-76c0a61e57c6","Type":"ContainerStarted","Data":"7da9256ce82e636903208cefdd2951eb59c36a4b2cfda31555e1145da1299bb5"} Jul 03 20:42:12.134673 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.134640 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqll" event={"ID":"c7c1946a-264d-4401-b552-12914faa5d86","Type":"ContainerStarted","Data":"57141a823a5336e389f8ac31c0349177a63ba9764c8b4abe5287f11cd284d0b6"} Jul 03 20:42:12.136413 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.136384 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bf8ff5f7-bkcjt" event={"ID":"7c900818-d7f5-4427-bbe0-1746c8a31487","Type":"ContainerStarted","Data":"70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca"} Jul 03 20:42:12.138786 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.138749 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:12.151196 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.151148 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8mqll" podStartSLOduration=33.783038804 podStartE2EDuration="52.151134501s" totalImagesPullingTime="18.368095697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 20:41:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:12.149970179 +0000 UTC m=+303.647537839" watchObservedRunningTime="2026-07-03 20:42:12.151134501 +0000 UTC m=+303.648702160" Jul 03 20:42:12.168134 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.168073 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86bf8ff5f7-bkcjt" podStartSLOduration=8.063636842 podStartE2EDuration="14.168051044s" totalImagesPullingTime="6.104414202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:12.165344677 +0000 UTC m=+303.662912329" watchObservedRunningTime="2026-07-03 20:42:12.168051044 +0000 UTC m=+303.665618708" Jul 03 20:42:12.334266 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:12.334134 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-85697c47cc-gdchw"] Jul 03 20:42:12.337346 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:42:12.337313 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a52ff3_7341_498e_a312_aa464d65aa34.slice/crio-c1fe2f8e7c5988b674261ad644490efdddf7b6bff4125f8adb7545cb0f33c963 WatchSource:0}: Error finding container c1fe2f8e7c5988b674261ad644490efdddf7b6bff4125f8adb7545cb0f33c963: Status 404 returned error can't find the container with id c1fe2f8e7c5988b674261ad644490efdddf7b6bff4125f8adb7545cb0f33c963 Jul 03 20:42:13.142274 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:13.142235 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" event={"ID":"f8a52ff3-7341-498e-a312-aa464d65aa34","Type":"ContainerStarted","Data":"c1fe2f8e7c5988b674261ad644490efdddf7b6bff4125f8adb7545cb0f33c963"} Jul 03 20:42:14.147449 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.147417 2558 generic.go:356] "Generic (PLEG): container finished" podID="14ee3481-359b-443f-8dad-76c0a61e57c6" containerID="fcf539077f0aa894cecfbfe26daf50f3b39dad5404b84811a2b31add16492698" exitCode=0 Jul 03 20:42:14.147839 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.147509 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26wsk" event={"ID":"14ee3481-359b-443f-8dad-76c0a61e57c6","Type":"ContainerDied","Data":"fcf539077f0aa894cecfbfe26daf50f3b39dad5404b84811a2b31add16492698"} Jul 03 20:42:14.637387 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.637355 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bf8ff5f7-bkcjt"] Jul 03 20:42:14.663252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.663222 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f4b5f6596-dc9zs"] Jul 03 20:42:14.680825 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.680798 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4b5f6596-dc9zs"] Jul 03 20:42:14.680968 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.680879 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.757714 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.757677 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-oauth-config\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.757902 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.757723 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-config\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.757902 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.757755 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ln5z\" (UniqueName: \"kubernetes.io/projected/c0f5805e-bb70-454c-95a9-690bd7ad2daf-kube-api-access-4ln5z\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.757902 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.757802 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-service-ca\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.757902 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.757852 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-oauth-serving-cert\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.758117 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.757923 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-serving-cert\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.758117 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.757979 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-trusted-ca-bundle\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.859498 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.859452 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-service-ca\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.859679 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.859543 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-oauth-serving-cert\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.859679 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.859596 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-serving-cert\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.859679 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.859640 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-trusted-ca-bundle\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.859875 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.859688 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-oauth-config\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.859875 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.859716 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-config\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.859875 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.859753 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ln5z\" (UniqueName: \"kubernetes.io/projected/c0f5805e-bb70-454c-95a9-690bd7ad2daf-kube-api-access-4ln5z\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.860701 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.860671 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-service-ca\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.860840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.860700 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-oauth-serving-cert\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.860909 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.860865 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-config\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.860909 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.860891 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-trusted-ca-bundle\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.862900 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.862879 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-serving-cert\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.863010 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.862943 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-oauth-config\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.867671 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.867645 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ln5z\" (UniqueName: \"kubernetes.io/projected/c0f5805e-bb70-454c-95a9-690bd7ad2daf-kube-api-access-4ln5z\") pod \"console-7f4b5f6596-dc9zs\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:14.992357 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:14.992286 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:15.139228 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:15.139203 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f4b5f6596-dc9zs"] Jul 03 20:42:15.152648 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:15.152622 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26wsk" event={"ID":"14ee3481-359b-443f-8dad-76c0a61e57c6","Type":"ContainerStarted","Data":"b5fdf9522ce3a6ca1a05d6efa74d11f8457e54c13833694194d9b0b2d9d2fc73"} Jul 03 20:42:15.153054 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:15.152673 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-26wsk" event={"ID":"14ee3481-359b-443f-8dad-76c0a61e57c6","Type":"ContainerStarted","Data":"79b65838c41c105760674e7d4f05bced15e548024eb47376d02c908774f2aa59"} Jul 03 20:42:15.171007 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:15.170967 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-26wsk" podStartSLOduration=4.4999722890000005 podStartE2EDuration="6.17095336s" totalImagesPullingTime="1.564057526s" totalInitContainerRuntime="106.923545ms" isStatefulPod=true podCreationTimestamp="2026-07-03 20:42:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:15.169784692 +0000 UTC m=+306.667352343" watchObservedRunningTime="2026-07-03 20:42:15.17095336 +0000 UTC m=+306.668521030" Jul 03 20:42:15.257223 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:42:15.257166 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f5805e_bb70_454c_95a9_690bd7ad2daf.slice/crio-b3caeb768533dbf46d0a25238ac01db4682ff22b2526d1ecf7999271d7edc5f9 WatchSource:0}: Error finding container b3caeb768533dbf46d0a25238ac01db4682ff22b2526d1ecf7999271d7edc5f9: Status 404 returned error can't find the container with id b3caeb768533dbf46d0a25238ac01db4682ff22b2526d1ecf7999271d7edc5f9 Jul 03 20:42:16.099059 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:16.099035 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rpcz2" Jul 03 20:42:16.159937 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:16.159904 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" event={"ID":"f8a52ff3-7341-498e-a312-aa464d65aa34","Type":"ContainerStarted","Data":"cedf249780db46c9a3771a466684563e5247140958bfc2491f9fae62620a0a98"} Jul 03 20:42:16.160297 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:16.159974 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" event={"ID":"f8a52ff3-7341-498e-a312-aa464d65aa34","Type":"ContainerStarted","Data":"01b68e6f9991461c01ae9fd3c7511a7564e46d78c170bf7c5ec0ccc2241a5779"} Jul 03 20:42:16.160297 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:16.160003 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" event={"ID":"f8a52ff3-7341-498e-a312-aa464d65aa34","Type":"ContainerStarted","Data":"1aaf4ae2e0f84ee28099ab07f749f2a10ceba6dcddd53700d62f89d3d24ad2e5"} Jul 03 20:42:16.161816 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:16.161788 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4b5f6596-dc9zs" event={"ID":"c0f5805e-bb70-454c-95a9-690bd7ad2daf","Type":"ContainerStarted","Data":"deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4"} Jul 03 20:42:16.161944 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:16.161833 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4b5f6596-dc9zs" event={"ID":"c0f5805e-bb70-454c-95a9-690bd7ad2daf","Type":"ContainerStarted","Data":"b3caeb768533dbf46d0a25238ac01db4682ff22b2526d1ecf7999271d7edc5f9"} Jul 03 20:42:16.178249 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:16.178208 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f4b5f6596-dc9zs" podStartSLOduration=2.178192344 podStartE2EDuration="2.178192344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:16.176828983 +0000 UTC m=+307.674396661" watchObservedRunningTime="2026-07-03 20:42:16.178192344 +0000 UTC m=+307.675760004" Jul 03 20:42:17.168587 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:17.168560 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" event={"ID":"f8a52ff3-7341-498e-a312-aa464d65aa34","Type":"ContainerStarted","Data":"c4022cf5c027497a4fd8a962fd8a9d68e2794c1f0a2072b6bf768d345fe69669"} Jul 03 20:42:17.169030 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:17.168644 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" event={"ID":"f8a52ff3-7341-498e-a312-aa464d65aa34","Type":"ContainerStarted","Data":"b71a0f9c3acdc77464a86b41c0d71ae54cf4a6c6ca6efdd4e8d268b81bab5829"} Jul 03 20:42:18.045626 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:18.045596 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:42:18.175095 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:18.175062 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" event={"ID":"f8a52ff3-7341-498e-a312-aa464d65aa34","Type":"ContainerStarted","Data":"89e07d32eb0dde9612667c854d19ffe47bbe6c468a94944c0e7285a14bc392a7"} Jul 03 20:42:18.175509 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:18.175184 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:18.200812 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:18.199722 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" podStartSLOduration=2.910601759 podStartE2EDuration="7.199703575s" totalImagesPullingTime="4.289101816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:42:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:18.196212467 +0000 UTC m=+309.693780128" watchObservedRunningTime="2026-07-03 20:42:18.199703575 +0000 UTC m=+309.697271241" Jul 03 20:42:18.912676 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:18.912649 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:42:19.274875 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.274794 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4b5f6596-dc9zs"] Jul 03 20:42:19.300087 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.300059 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79787d6886-swvl9"] Jul 03 20:42:19.318985 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.318957 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79787d6886-swvl9"] Jul 03 20:42:19.319139 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.319077 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.399022 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.398987 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-config\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.399187 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.399031 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wr4r\" (UniqueName: \"kubernetes.io/projected/22ab6509-1ed4-4cd0-b20c-c5307ab19494-kube-api-access-8wr4r\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.399187 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.399112 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-oauth-config\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.399187 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.399137 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-oauth-serving-cert\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.399187 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.399161 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-service-ca\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.399340 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.399223 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-serving-cert\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.399340 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.399319 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-trusted-ca-bundle\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.500333 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.500299 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-oauth-config\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.500333 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.500335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-oauth-serving-cert\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.500577 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.500360 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-service-ca\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.500577 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.500378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-serving-cert\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.500577 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.500450 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-trusted-ca-bundle\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.500577 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.500467 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-config\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.500577 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.500486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wr4r\" (UniqueName: \"kubernetes.io/projected/22ab6509-1ed4-4cd0-b20c-c5307ab19494-kube-api-access-8wr4r\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.501207 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.501182 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-oauth-serving-cert\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.501293 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.501265 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-config\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.501293 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.501272 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-service-ca\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.501634 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.501609 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-trusted-ca-bundle\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.502903 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.502883 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-oauth-config\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.503011 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.502994 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-serving-cert\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.507631 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.507615 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wr4r\" (UniqueName: \"kubernetes.io/projected/22ab6509-1ed4-4cd0-b20c-c5307ab19494-kube-api-access-8wr4r\") pod \"console-79787d6886-swvl9\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.628786 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.628750 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:19.779510 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.779489 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79787d6886-swvl9"] Jul 03 20:42:19.781749 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:42:19.781725 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ab6509_1ed4_4cd0_b20c_c5307ab19494.slice/crio-bfa378331d533ff9830f62d3b807d9e302bc61607e69f2440dd249e34856d3df WatchSource:0}: Error finding container bfa378331d533ff9830f62d3b807d9e302bc61607e69f2440dd249e34856d3df: Status 404 returned error can't find the container with id bfa378331d533ff9830f62d3b807d9e302bc61607e69f2440dd249e34856d3df Jul 03 20:42:19.844940 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:19.844910 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:42:20.183719 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:20.183640 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79787d6886-swvl9" event={"ID":"22ab6509-1ed4-4cd0-b20c-c5307ab19494","Type":"ContainerStarted","Data":"4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8"} Jul 03 20:42:20.183719 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:20.183685 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79787d6886-swvl9" event={"ID":"22ab6509-1ed4-4cd0-b20c-c5307ab19494","Type":"ContainerStarted","Data":"bfa378331d533ff9830f62d3b807d9e302bc61607e69f2440dd249e34856d3df"} Jul 03 20:42:20.202342 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:20.202300 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79787d6886-swvl9" podStartSLOduration=1.202285237 podStartE2EDuration="1.202285237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:42:20.200650215 +0000 UTC m=+311.698217874" watchObservedRunningTime="2026-07-03 20:42:20.202285237 +0000 UTC m=+311.699852897" Jul 03 20:42:24.184719 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:24.184685 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-85697c47cc-gdchw" Jul 03 20:42:24.992855 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:24.992821 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:29.629528 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:29.629492 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:29.629995 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:29.629562 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:29.634890 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:29.634868 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:30.224759 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:30.224729 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:42:34.891720 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:34.891674 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" containerName="registry" containerID="cri-o://2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0" gracePeriod=30 Jul 03 20:42:35.135653 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.135632 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:42:35.233966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.233882 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-bound-sa-token\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.233966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.233915 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-trusted-ca\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.233966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.233940 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-certificates\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.233966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.233965 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-installation-pull-secrets\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.234289 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234087 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-image-registry-private-configuration\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.234289 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234155 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.234289 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234203 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d1704bd-4282-4647-99e0-498fa8c9a70e-ca-trust-extracted\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.234289 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234239 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2stjg\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-kube-api-access-2stjg\") pod \"0d1704bd-4282-4647-99e0-498fa8c9a70e\" (UID: \"0d1704bd-4282-4647-99e0-498fa8c9a70e\") " Jul 03 20:42:35.234495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234378 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:35.234495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234383 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:35.234669 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234646 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-trusted-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.234743 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.234679 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-certificates\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.237085 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.237006 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:42:35.237205 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.237092 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:42:35.237271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.237233 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:42:35.237332 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.237289 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-kube-api-access-2stjg" (OuterVolumeSpecName: "kube-api-access-2stjg") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "kube-api-access-2stjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:42:35.237332 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.237301 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:42:35.241564 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.241542 2558 generic.go:356] "Generic (PLEG): container finished" podID="6638ddc3-acba-40eb-b82a-3ffeb3799103" containerID="050986c0f7e67fdc67d96ad08531bdbe4c38b5568e5982126681fc774dabdd71" exitCode=0 Jul 03 20:42:35.241680 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.241618 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" event={"ID":"6638ddc3-acba-40eb-b82a-3ffeb3799103","Type":"ContainerDied","Data":"050986c0f7e67fdc67d96ad08531bdbe4c38b5568e5982126681fc774dabdd71"} Jul 03 20:42:35.242080 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.242005 2558 scope.go:117] "RemoveContainer" containerID="050986c0f7e67fdc67d96ad08531bdbe4c38b5568e5982126681fc774dabdd71" Jul 03 20:42:35.243481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.243421 2558 generic.go:356] "Generic (PLEG): container finished" podID="0d1704bd-4282-4647-99e0-498fa8c9a70e" containerID="2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0" exitCode=0 Jul 03 20:42:35.243481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.243455 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" event={"ID":"0d1704bd-4282-4647-99e0-498fa8c9a70e","Type":"ContainerDied","Data":"2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0"} Jul 03 20:42:35.243481 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.243475 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" Jul 03 20:42:35.243649 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.243497 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d5859fc4f-5gb28" event={"ID":"0d1704bd-4282-4647-99e0-498fa8c9a70e","Type":"ContainerDied","Data":"3f86015bc614361b2a5135cb55dc3171def76d1f061d20ca2f287c08e861cbc9"} Jul 03 20:42:35.244491 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.244369 2558 scope.go:117] "RemoveContainer" containerID="2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0" Jul 03 20:42:35.244939 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.244915 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1704bd-4282-4647-99e0-498fa8c9a70e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0d1704bd-4282-4647-99e0-498fa8c9a70e" (UID: "0d1704bd-4282-4647-99e0-498fa8c9a70e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:42:35.256947 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.256864 2558 scope.go:117] "RemoveContainer" containerID="2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0" Jul 03 20:42:35.257340 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:35.257318 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0\": container with ID starting with 2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0 not found: ID does not exist" containerID="2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0" Jul 03 20:42:35.257416 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.257350 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0"} err="failed to get container status \"2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0\": rpc error: code = NotFound desc = could not find container \"2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0\": container with ID starting with 2e0ff076e98684e4b49b0a909759689790adc268f1d80d65d3f9baf2a6c902a0 not found: ID does not exist" Jul 03 20:42:35.335243 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.335209 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-bound-sa-token\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.335243 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.335237 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-installation-pull-secrets\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.335243 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.335250 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0d1704bd-4282-4647-99e0-498fa8c9a70e-image-registry-private-configuration\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.335473 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.335263 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-registry-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.335473 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.335277 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0d1704bd-4282-4647-99e0-498fa8c9a70e-ca-trust-extracted\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.335473 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.335288 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2stjg\" (UniqueName: \"kubernetes.io/projected/0d1704bd-4282-4647-99e0-498fa8c9a70e-kube-api-access-2stjg\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:35.565234 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.565203 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d5859fc4f-5gb28"] Jul 03 20:42:35.568809 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:35.568788 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d5859fc4f-5gb28"] Jul 03 20:42:36.249014 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:36.248978 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-vdzf8" event={"ID":"6638ddc3-acba-40eb-b82a-3ffeb3799103","Type":"ContainerStarted","Data":"70c4b68f0ebd7b9dd0bb46a3514f83b93ef2a4553c76a74181725108fcc6c548"} Jul 03 20:42:37.050562 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:37.050526 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" path="/var/lib/kubelet/pods/0d1704bd-4282-4647-99e0-498fa8c9a70e/volumes" Jul 03 20:42:39.667729 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:39.667669 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86bf8ff5f7-bkcjt" podUID="7c900818-d7f5-4427-bbe0-1746c8a31487" containerName="console" containerID="cri-o://70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca" gracePeriod=15 Jul 03 20:42:39.945170 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:39.945110 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:42:40.075495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075462 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-oauth-config\") pod \"7c900818-d7f5-4427-bbe0-1746c8a31487\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " Jul 03 20:42:40.075495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075499 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-trusted-ca-bundle\") pod \"7c900818-d7f5-4427-bbe0-1746c8a31487\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " Jul 03 20:42:40.075680 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075519 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-oauth-serving-cert\") pod \"7c900818-d7f5-4427-bbe0-1746c8a31487\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " Jul 03 20:42:40.075680 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075554 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r8lx\" (UniqueName: \"kubernetes.io/projected/7c900818-d7f5-4427-bbe0-1746c8a31487-kube-api-access-4r8lx\") pod \"7c900818-d7f5-4427-bbe0-1746c8a31487\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " Jul 03 20:42:40.075680 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075611 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-serving-cert\") pod \"7c900818-d7f5-4427-bbe0-1746c8a31487\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " Jul 03 20:42:40.075680 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075656 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-console-config\") pod \"7c900818-d7f5-4427-bbe0-1746c8a31487\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " Jul 03 20:42:40.075880 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075709 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-service-ca\") pod \"7c900818-d7f5-4427-bbe0-1746c8a31487\" (UID: \"7c900818-d7f5-4427-bbe0-1746c8a31487\") " Jul 03 20:42:40.076032 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.075991 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7c900818-d7f5-4427-bbe0-1746c8a31487" (UID: "7c900818-d7f5-4427-bbe0-1746c8a31487"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:40.076187 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.076080 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-console-config" (OuterVolumeSpecName: "console-config") pod "7c900818-d7f5-4427-bbe0-1746c8a31487" (UID: "7c900818-d7f5-4427-bbe0-1746c8a31487"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:40.076301 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.076094 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7c900818-d7f5-4427-bbe0-1746c8a31487" (UID: "7c900818-d7f5-4427-bbe0-1746c8a31487"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:40.076301 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.076246 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-service-ca" (OuterVolumeSpecName: "service-ca") pod "7c900818-d7f5-4427-bbe0-1746c8a31487" (UID: "7c900818-d7f5-4427-bbe0-1746c8a31487"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:40.077797 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.077758 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7c900818-d7f5-4427-bbe0-1746c8a31487" (UID: "7c900818-d7f5-4427-bbe0-1746c8a31487"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:42:40.077879 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.077841 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c900818-d7f5-4427-bbe0-1746c8a31487-kube-api-access-4r8lx" (OuterVolumeSpecName: "kube-api-access-4r8lx") pod "7c900818-d7f5-4427-bbe0-1746c8a31487" (UID: "7c900818-d7f5-4427-bbe0-1746c8a31487"). InnerVolumeSpecName "kube-api-access-4r8lx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:42:40.077920 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.077878 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7c900818-d7f5-4427-bbe0-1746c8a31487" (UID: "7c900818-d7f5-4427-bbe0-1746c8a31487"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:42:40.177238 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.177212 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:40.177238 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.177235 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-oauth-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:40.177238 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.177244 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-trusted-ca-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:40.177406 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.177253 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-oauth-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:40.177406 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.177261 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r8lx\" (UniqueName: \"kubernetes.io/projected/7c900818-d7f5-4427-bbe0-1746c8a31487-kube-api-access-4r8lx\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:40.177406 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.177274 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c900818-d7f5-4427-bbe0-1746c8a31487-console-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:40.177406 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.177286 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c900818-d7f5-4427-bbe0-1746c8a31487-console-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:40.265984 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.265914 2558 generic.go:356] "Generic (PLEG): container finished" podID="7c900818-d7f5-4427-bbe0-1746c8a31487" containerID="70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca" exitCode=0 Jul 03 20:42:40.265984 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.265960 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bf8ff5f7-bkcjt" Jul 03 20:42:40.265984 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.265970 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bf8ff5f7-bkcjt" event={"ID":"7c900818-d7f5-4427-bbe0-1746c8a31487","Type":"ContainerDied","Data":"70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca"} Jul 03 20:42:40.266170 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.266009 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bf8ff5f7-bkcjt" event={"ID":"7c900818-d7f5-4427-bbe0-1746c8a31487","Type":"ContainerDied","Data":"1b073dadc08a35b705c53048ff21bef91cd3cb994e9fcff705b998a6df63480d"} Jul 03 20:42:40.266170 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.266028 2558 scope.go:117] "RemoveContainer" containerID="70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca" Jul 03 20:42:40.279937 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.279922 2558 scope.go:117] "RemoveContainer" containerID="70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca" Jul 03 20:42:40.280183 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:40.280163 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca\": container with ID starting with 70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca not found: ID does not exist" containerID="70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca" Jul 03 20:42:40.280234 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.280191 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca"} err="failed to get container status \"70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca\": rpc error: code = NotFound desc = could not find container \"70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca\": container with ID starting with 70b9b7b4fba8b0eb4cb8b7e67372a57d8bf25c0f6ce4f253414c33e670e632ca not found: ID does not exist" Jul 03 20:42:40.286738 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.286717 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bf8ff5f7-bkcjt"] Jul 03 20:42:40.290057 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:40.290038 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86bf8ff5f7-bkcjt"] Jul 03 20:42:41.055006 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:41.053083 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c900818-d7f5-4427-bbe0-1746c8a31487" path="/var/lib/kubelet/pods/7c900818-d7f5-4427-bbe0-1746c8a31487/volumes" Jul 03 20:42:44.294266 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.294210 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7f4b5f6596-dc9zs" podUID="c0f5805e-bb70-454c-95a9-690bd7ad2daf" containerName="console" containerID="cri-o://deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4" gracePeriod=15 Jul 03 20:42:44.566639 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.566583 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:44.715854 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.715816 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ln5z\" (UniqueName: \"kubernetes.io/projected/c0f5805e-bb70-454c-95a9-690bd7ad2daf-kube-api-access-4ln5z\") pod \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " Jul 03 20:42:44.716052 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.715875 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-serving-cert\") pod \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " Jul 03 20:42:44.716052 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.715926 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-service-ca\") pod \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " Jul 03 20:42:44.716052 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.715958 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-oauth-serving-cert\") pod \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " Jul 03 20:42:44.716203 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.716074 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-oauth-config\") pod \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " Jul 03 20:42:44.716203 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.716108 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-config\") pod \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " Jul 03 20:42:44.716203 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.716125 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-trusted-ca-bundle\") pod \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\" (UID: \"c0f5805e-bb70-454c-95a9-690bd7ad2daf\") " Jul 03 20:42:44.716424 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.716402 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c0f5805e-bb70-454c-95a9-690bd7ad2daf" (UID: "c0f5805e-bb70-454c-95a9-690bd7ad2daf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:44.716485 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.716414 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-service-ca" (OuterVolumeSpecName: "service-ca") pod "c0f5805e-bb70-454c-95a9-690bd7ad2daf" (UID: "c0f5805e-bb70-454c-95a9-690bd7ad2daf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:44.716534 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.716483 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-config" (OuterVolumeSpecName: "console-config") pod "c0f5805e-bb70-454c-95a9-690bd7ad2daf" (UID: "c0f5805e-bb70-454c-95a9-690bd7ad2daf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:44.716712 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.716695 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c0f5805e-bb70-454c-95a9-690bd7ad2daf" (UID: "c0f5805e-bb70-454c-95a9-690bd7ad2daf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:42:44.718043 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.718023 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f5805e-bb70-454c-95a9-690bd7ad2daf-kube-api-access-4ln5z" (OuterVolumeSpecName: "kube-api-access-4ln5z") pod "c0f5805e-bb70-454c-95a9-690bd7ad2daf" (UID: "c0f5805e-bb70-454c-95a9-690bd7ad2daf"). InnerVolumeSpecName "kube-api-access-4ln5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:42:44.718504 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.718486 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c0f5805e-bb70-454c-95a9-690bd7ad2daf" (UID: "c0f5805e-bb70-454c-95a9-690bd7ad2daf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:42:44.718554 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.718509 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c0f5805e-bb70-454c-95a9-690bd7ad2daf" (UID: "c0f5805e-bb70-454c-95a9-690bd7ad2daf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:42:44.817630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.817553 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-oauth-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:44.817630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.817579 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:44.817630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.817591 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-trusted-ca-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:44.817630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.817603 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ln5z\" (UniqueName: \"kubernetes.io/projected/c0f5805e-bb70-454c-95a9-690bd7ad2daf-kube-api-access-4ln5z\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:44.817630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.817614 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0f5805e-bb70-454c-95a9-690bd7ad2daf-console-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:44.817630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.817628 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:44.817921 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:44.817640 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0f5805e-bb70-454c-95a9-690bd7ad2daf-oauth-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:42:45.286485 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.286402 2558 generic.go:356] "Generic (PLEG): container finished" podID="c0f5805e-bb70-454c-95a9-690bd7ad2daf" containerID="deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4" exitCode=0 Jul 03 20:42:45.286485 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.286446 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f4b5f6596-dc9zs" Jul 03 20:42:45.286661 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.286485 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4b5f6596-dc9zs" event={"ID":"c0f5805e-bb70-454c-95a9-690bd7ad2daf","Type":"ContainerDied","Data":"deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4"} Jul 03 20:42:45.286661 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.286529 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f4b5f6596-dc9zs" event={"ID":"c0f5805e-bb70-454c-95a9-690bd7ad2daf","Type":"ContainerDied","Data":"b3caeb768533dbf46d0a25238ac01db4682ff22b2526d1ecf7999271d7edc5f9"} Jul 03 20:42:45.286661 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.286553 2558 scope.go:117] "RemoveContainer" containerID="deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4" Jul 03 20:42:45.288146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.288124 2558 generic.go:356] "Generic (PLEG): container finished" podID="0e0b6df9-93cb-4d46-b0a1-c7318a298782" containerID="d0e241c4c6af1be13d2d03003349a684e4cad180d0b4e417a00b0db7b4250142" exitCode=0 Jul 03 20:42:45.288273 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.288162 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" event={"ID":"0e0b6df9-93cb-4d46-b0a1-c7318a298782","Type":"ContainerDied","Data":"d0e241c4c6af1be13d2d03003349a684e4cad180d0b4e417a00b0db7b4250142"} Jul 03 20:42:45.288506 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.288487 2558 scope.go:117] "RemoveContainer" containerID="d0e241c4c6af1be13d2d03003349a684e4cad180d0b4e417a00b0db7b4250142" Jul 03 20:42:45.295049 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.294848 2558 scope.go:117] "RemoveContainer" containerID="deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4" Jul 03 20:42:45.295261 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:42:45.295091 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4\": container with ID starting with deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4 not found: ID does not exist" containerID="deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4" Jul 03 20:42:45.295261 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.295113 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4"} err="failed to get container status \"deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4\": rpc error: code = NotFound desc = could not find container \"deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4\": container with ID starting with deab6f419233a2a11e282e24b035c74c57e180ad8b5d2c4528368359cf504bf4 not found: ID does not exist" Jul 03 20:42:45.300965 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.300945 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f4b5f6596-dc9zs"] Jul 03 20:42:45.303063 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:45.303041 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f4b5f6596-dc9zs"] Jul 03 20:42:46.293821 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:46.293783 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-5dbt7" event={"ID":"0e0b6df9-93cb-4d46-b0a1-c7318a298782","Type":"ContainerStarted","Data":"d56d2483d74b1cdc7401c069b9f5383216120302099cc85f104445a07b2da63c"} Jul 03 20:42:47.050951 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:42:47.050914 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f5805e-bb70-454c-95a9-690bd7ad2daf" path="/var/lib/kubelet/pods/c0f5805e-bb70-454c-95a9-690bd7ad2daf/volumes" Jul 03 20:43:21.914309 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:21.914267 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:43:21.917057 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:21.917029 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3dec259-8bc3-4826-96aa-924c10061798-metrics-certs\") pod \"network-metrics-daemon-85jm6\" (UID: \"e3dec259-8bc3-4826-96aa-924c10061798\") " pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:43:21.949321 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:21.949289 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4xwrm\"" Jul 03 20:43:21.956872 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:21.956851 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-85jm6" Jul 03 20:43:22.083798 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:22.083773 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-85jm6"] Jul 03 20:43:22.087365 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:43:22.087342 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3dec259_8bc3_4826_96aa_924c10061798.slice/crio-434b509a859b15a4b0a46112868fa75cd0d4f256310367c69fd9ee60b3bbfda1 WatchSource:0}: Error finding container 434b509a859b15a4b0a46112868fa75cd0d4f256310367c69fd9ee60b3bbfda1: Status 404 returned error can't find the container with id 434b509a859b15a4b0a46112868fa75cd0d4f256310367c69fd9ee60b3bbfda1 Jul 03 20:43:22.428284 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:22.428255 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-85jm6" event={"ID":"e3dec259-8bc3-4826-96aa-924c10061798","Type":"ContainerStarted","Data":"434b509a859b15a4b0a46112868fa75cd0d4f256310367c69fd9ee60b3bbfda1"} Jul 03 20:43:23.433670 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:23.433579 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-85jm6" event={"ID":"e3dec259-8bc3-4826-96aa-924c10061798","Type":"ContainerStarted","Data":"8e58f17c2b791582cf1753ca68a9a92a5bff391474df984c7266da1a1d829d7b"} Jul 03 20:43:23.433670 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:23.433636 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-85jm6" event={"ID":"e3dec259-8bc3-4826-96aa-924c10061798","Type":"ContainerStarted","Data":"441e6fe1fb0c9d3d035560b02742f58dabd4fb653fd68cf477a8ac9848665a50"} Jul 03 20:43:23.450170 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:23.450123 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-85jm6" podStartSLOduration=373.451662001 podStartE2EDuration="6m14.45009165s" totalImagesPullingTime="998.429649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:43:23.449925943 +0000 UTC m=+374.947493604" watchObservedRunningTime="2026-07-03 20:43:23.45009165 +0000 UTC m=+374.947659309" Jul 03 20:43:41.313904 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.313874 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54659b7845-rchdr"] Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314156 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f5805e-bb70-454c-95a9-690bd7ad2daf" containerName="console" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314169 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f5805e-bb70-454c-95a9-690bd7ad2daf" containerName="console" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314183 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c900818-d7f5-4427-bbe0-1746c8a31487" containerName="console" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314189 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c900818-d7f5-4427-bbe0-1746c8a31487" containerName="console" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314201 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" containerName="registry" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314207 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" containerName="registry" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314258 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c900818-d7f5-4427-bbe0-1746c8a31487" containerName="console" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314266 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f5805e-bb70-454c-95a9-690bd7ad2daf" containerName="console" Jul 03 20:43:41.314379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.314275 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d1704bd-4282-4647-99e0-498fa8c9a70e" containerName="registry" Jul 03 20:43:41.316099 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.316083 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.331325 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.331294 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54659b7845-rchdr"] Jul 03 20:43:41.466257 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.466219 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-serving-cert\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.466257 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.466256 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-oauth-serving-cert\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.466459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.466273 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xpfc\" (UniqueName: \"kubernetes.io/projected/0cf2eedf-3d0a-4635-b41a-77b07184dd23-kube-api-access-9xpfc\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.466459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.466312 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-trusted-ca-bundle\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.466459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.466348 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-oauth-config\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.466459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.466377 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-config\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.466459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.466436 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-service-ca\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.567823 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.567712 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-trusted-ca-bundle\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.567823 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.567753 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-oauth-config\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568066 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.567945 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-config\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568066 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.567994 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-service-ca\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568066 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.568033 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-serving-cert\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568066 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.568050 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-oauth-serving-cert\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568066 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.568068 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xpfc\" (UniqueName: \"kubernetes.io/projected/0cf2eedf-3d0a-4635-b41a-77b07184dd23-kube-api-access-9xpfc\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568811 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.568758 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-trusted-ca-bundle\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568914 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.568824 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-service-ca\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.568914 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.568885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-oauth-serving-cert\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.569258 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.569234 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-config\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.570371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.570352 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-oauth-config\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.570451 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.570404 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-serving-cert\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.578582 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.578555 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xpfc\" (UniqueName: \"kubernetes.io/projected/0cf2eedf-3d0a-4635-b41a-77b07184dd23-kube-api-access-9xpfc\") pod \"console-54659b7845-rchdr\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.625359 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.625324 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:41.753382 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:41.753355 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54659b7845-rchdr"] Jul 03 20:43:41.755235 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:43:41.755206 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf2eedf_3d0a_4635_b41a_77b07184dd23.slice/crio-97dcbf766d8dfea0caf4ff5f662e6f67f14a02743a7f89052cea639682ddd339 WatchSource:0}: Error finding container 97dcbf766d8dfea0caf4ff5f662e6f67f14a02743a7f89052cea639682ddd339: Status 404 returned error can't find the container with id 97dcbf766d8dfea0caf4ff5f662e6f67f14a02743a7f89052cea639682ddd339 Jul 03 20:43:42.507985 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:42.507947 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54659b7845-rchdr" event={"ID":"0cf2eedf-3d0a-4635-b41a-77b07184dd23","Type":"ContainerStarted","Data":"0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d"} Jul 03 20:43:42.508370 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:42.508003 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54659b7845-rchdr" event={"ID":"0cf2eedf-3d0a-4635-b41a-77b07184dd23","Type":"ContainerStarted","Data":"97dcbf766d8dfea0caf4ff5f662e6f67f14a02743a7f89052cea639682ddd339"} Jul 03 20:43:42.532071 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:42.532016 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54659b7845-rchdr" podStartSLOduration=1.531999943 podStartE2EDuration="1.531999943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:43:42.530289461 +0000 UTC m=+394.027857137" watchObservedRunningTime="2026-07-03 20:43:42.531999943 +0000 UTC m=+394.029567893" Jul 03 20:43:51.625552 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:51.625514 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:51.626062 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:51.625574 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:51.630407 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:51.630385 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:52.549324 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:52.549299 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:43:52.600485 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:43:52.600456 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79787d6886-swvl9"] Jul 03 20:44:02.669648 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.669618 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6"] Jul 03 20:44:02.672019 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.672004 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.674716 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.674694 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 20:44:02.675682 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.675662 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 20:44:02.675806 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.675730 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-clrpt\"" Jul 03 20:44:02.684374 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.684350 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6"] Jul 03 20:44:02.726235 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.726210 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nkqd\" (UniqueName: \"kubernetes.io/projected/71e40c11-0e44-456d-b963-80aa609834b7-kube-api-access-4nkqd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.726360 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.726251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.726360 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.726271 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.827440 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.827405 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nkqd\" (UniqueName: \"kubernetes.io/projected/71e40c11-0e44-456d-b963-80aa609834b7-kube-api-access-4nkqd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.827620 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.827453 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.827687 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.827623 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.827877 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.827855 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.827948 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.827928 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.836520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.836499 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nkqd\" (UniqueName: \"kubernetes.io/projected/71e40c11-0e44-456d-b963-80aa609834b7-kube-api-access-4nkqd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:02.981271 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:02.981184 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:03.114997 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:03.114974 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6"] Jul 03 20:44:03.116922 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:44:03.116895 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e40c11_0e44_456d_b963_80aa609834b7.slice/crio-ff0bdb3d221307f2123a2eaecd0f457599b5de68555d1919b3f3796c82c4d908 WatchSource:0}: Error finding container ff0bdb3d221307f2123a2eaecd0f457599b5de68555d1919b3f3796c82c4d908: Status 404 returned error can't find the container with id ff0bdb3d221307f2123a2eaecd0f457599b5de68555d1919b3f3796c82c4d908 Jul 03 20:44:03.584309 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:03.584279 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" event={"ID":"71e40c11-0e44-456d-b963-80aa609834b7","Type":"ContainerStarted","Data":"ff0bdb3d221307f2123a2eaecd0f457599b5de68555d1919b3f3796c82c4d908"} Jul 03 20:44:09.610808 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:09.610753 2558 generic.go:356] "Generic (PLEG): container finished" podID="71e40c11-0e44-456d-b963-80aa609834b7" containerID="813902c09e03333f672ec1f593512bc32141e276c4f947511cafdd2ae338a2b5" exitCode=0 Jul 03 20:44:09.611209 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:09.610858 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" event={"ID":"71e40c11-0e44-456d-b963-80aa609834b7","Type":"ContainerDied","Data":"813902c09e03333f672ec1f593512bc32141e276c4f947511cafdd2ae338a2b5"} Jul 03 20:44:12.625305 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:12.625271 2558 generic.go:356] "Generic (PLEG): container finished" podID="71e40c11-0e44-456d-b963-80aa609834b7" containerID="e582e2f18daf6d0b278720f5cf260f272e6ff9bd6624f0018ad096f5371ac89d" exitCode=0 Jul 03 20:44:12.625753 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:12.625344 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" event={"ID":"71e40c11-0e44-456d-b963-80aa609834b7","Type":"ContainerDied","Data":"e582e2f18daf6d0b278720f5cf260f272e6ff9bd6624f0018ad096f5371ac89d"} Jul 03 20:44:17.625122 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:17.625063 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-79787d6886-swvl9" podUID="22ab6509-1ed4-4cd0-b20c-c5307ab19494" containerName="console" containerID="cri-o://4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8" gracePeriod=15 Jul 03 20:44:19.372882 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.372861 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:44:19.451728 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.451692 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-serving-cert\") pod \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " Jul 03 20:44:19.451914 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.451744 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-service-ca\") pod \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " Jul 03 20:44:19.451914 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.451780 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-oauth-serving-cert\") pod \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " Jul 03 20:44:19.451914 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.451807 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-config\") pod \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " Jul 03 20:44:19.452058 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452011 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-trusted-ca-bundle\") pod \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " Jul 03 20:44:19.452113 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452071 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-oauth-config\") pod \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " Jul 03 20:44:19.452171 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452116 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "22ab6509-1ed4-4cd0-b20c-c5307ab19494" (UID: "22ab6509-1ed4-4cd0-b20c-c5307ab19494"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:44:19.452171 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452139 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wr4r\" (UniqueName: \"kubernetes.io/projected/22ab6509-1ed4-4cd0-b20c-c5307ab19494-kube-api-access-8wr4r\") pod \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\" (UID: \"22ab6509-1ed4-4cd0-b20c-c5307ab19494\") " Jul 03 20:44:19.452371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452273 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-config" (OuterVolumeSpecName: "console-config") pod "22ab6509-1ed4-4cd0-b20c-c5307ab19494" (UID: "22ab6509-1ed4-4cd0-b20c-c5307ab19494"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:44:19.452371 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452325 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "22ab6509-1ed4-4cd0-b20c-c5307ab19494" (UID: "22ab6509-1ed4-4cd0-b20c-c5307ab19494"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:44:19.452541 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452522 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-oauth-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:19.452613 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452541 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:19.452613 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452556 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-trusted-ca-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:19.452613 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.452522 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-service-ca" (OuterVolumeSpecName: "service-ca") pod "22ab6509-1ed4-4cd0-b20c-c5307ab19494" (UID: "22ab6509-1ed4-4cd0-b20c-c5307ab19494"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:44:19.454111 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.454087 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "22ab6509-1ed4-4cd0-b20c-c5307ab19494" (UID: "22ab6509-1ed4-4cd0-b20c-c5307ab19494"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:44:19.454541 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.454514 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "22ab6509-1ed4-4cd0-b20c-c5307ab19494" (UID: "22ab6509-1ed4-4cd0-b20c-c5307ab19494"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:44:19.454621 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.454538 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ab6509-1ed4-4cd0-b20c-c5307ab19494-kube-api-access-8wr4r" (OuterVolumeSpecName: "kube-api-access-8wr4r") pod "22ab6509-1ed4-4cd0-b20c-c5307ab19494" (UID: "22ab6509-1ed4-4cd0-b20c-c5307ab19494"). InnerVolumeSpecName "kube-api-access-8wr4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:44:19.553516 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.553492 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wr4r\" (UniqueName: \"kubernetes.io/projected/22ab6509-1ed4-4cd0-b20c-c5307ab19494-kube-api-access-8wr4r\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:19.553516 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.553517 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:19.553665 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.553528 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22ab6509-1ed4-4cd0-b20c-c5307ab19494-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:19.553665 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.553536 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22ab6509-1ed4-4cd0-b20c-c5307ab19494-console-oauth-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:19.665196 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.665158 2558 generic.go:356] "Generic (PLEG): container finished" podID="71e40c11-0e44-456d-b963-80aa609834b7" containerID="d26a691034bbd3b1213acf23d3e711a8dc5f4fd2d182882076e3456a75f84405" exitCode=0 Jul 03 20:44:19.665308 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.665249 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" event={"ID":"71e40c11-0e44-456d-b963-80aa609834b7","Type":"ContainerDied","Data":"d26a691034bbd3b1213acf23d3e711a8dc5f4fd2d182882076e3456a75f84405"} Jul 03 20:44:19.666480 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.666453 2558 generic.go:356] "Generic (PLEG): container finished" podID="22ab6509-1ed4-4cd0-b20c-c5307ab19494" containerID="4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8" exitCode=0 Jul 03 20:44:19.666555 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.666489 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79787d6886-swvl9" Jul 03 20:44:19.666555 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.666524 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79787d6886-swvl9" event={"ID":"22ab6509-1ed4-4cd0-b20c-c5307ab19494","Type":"ContainerDied","Data":"4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8"} Jul 03 20:44:19.666638 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.666565 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79787d6886-swvl9" event={"ID":"22ab6509-1ed4-4cd0-b20c-c5307ab19494","Type":"ContainerDied","Data":"bfa378331d533ff9830f62d3b807d9e302bc61607e69f2440dd249e34856d3df"} Jul 03 20:44:19.666638 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.666589 2558 scope.go:117] "RemoveContainer" containerID="4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8" Jul 03 20:44:19.675373 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.675355 2558 scope.go:117] "RemoveContainer" containerID="4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8" Jul 03 20:44:19.675634 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:44:19.675615 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8\": container with ID starting with 4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8 not found: ID does not exist" containerID="4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8" Jul 03 20:44:19.675704 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.675641 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8"} err="failed to get container status \"4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8\": rpc error: code = NotFound desc = could not find container \"4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8\": container with ID starting with 4774ac2439af73e8b63be24426916468c9ab741e03c2efbe1aab2d858c3faad8 not found: ID does not exist" Jul 03 20:44:19.696169 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.696151 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79787d6886-swvl9"] Jul 03 20:44:19.702415 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:19.702395 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79787d6886-swvl9"] Jul 03 20:44:20.799338 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.799316 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:20.864994 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.864967 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-util\") pod \"71e40c11-0e44-456d-b963-80aa609834b7\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " Jul 03 20:44:20.865143 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.865007 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-bundle\") pod \"71e40c11-0e44-456d-b963-80aa609834b7\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " Jul 03 20:44:20.865143 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.865057 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nkqd\" (UniqueName: \"kubernetes.io/projected/71e40c11-0e44-456d-b963-80aa609834b7-kube-api-access-4nkqd\") pod \"71e40c11-0e44-456d-b963-80aa609834b7\" (UID: \"71e40c11-0e44-456d-b963-80aa609834b7\") " Jul 03 20:44:20.865712 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.865674 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-bundle" (OuterVolumeSpecName: "bundle") pod "71e40c11-0e44-456d-b963-80aa609834b7" (UID: "71e40c11-0e44-456d-b963-80aa609834b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:44:20.867315 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.867285 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e40c11-0e44-456d-b963-80aa609834b7-kube-api-access-4nkqd" (OuterVolumeSpecName: "kube-api-access-4nkqd") pod "71e40c11-0e44-456d-b963-80aa609834b7" (UID: "71e40c11-0e44-456d-b963-80aa609834b7"). InnerVolumeSpecName "kube-api-access-4nkqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:44:20.868874 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.868852 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-util" (OuterVolumeSpecName: "util") pod "71e40c11-0e44-456d-b963-80aa609834b7" (UID: "71e40c11-0e44-456d-b963-80aa609834b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:44:20.965940 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.965852 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nkqd\" (UniqueName: \"kubernetes.io/projected/71e40c11-0e44-456d-b963-80aa609834b7-kube-api-access-4nkqd\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:20.965940 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.965882 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-util\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:20.965940 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:20.965892 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71e40c11-0e44-456d-b963-80aa609834b7-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:21.051078 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:21.051048 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22ab6509-1ed4-4cd0-b20c-c5307ab19494" path="/var/lib/kubelet/pods/22ab6509-1ed4-4cd0-b20c-c5307ab19494/volumes" Jul 03 20:44:21.677774 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:21.677746 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" Jul 03 20:44:21.677774 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:21.677751 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409zbl6" event={"ID":"71e40c11-0e44-456d-b963-80aa609834b7","Type":"ContainerDied","Data":"ff0bdb3d221307f2123a2eaecd0f457599b5de68555d1919b3f3796c82c4d908"} Jul 03 20:44:21.677960 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:21.677807 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff0bdb3d221307f2123a2eaecd0f457599b5de68555d1919b3f3796c82c4d908" Jul 03 20:44:26.121055 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121022 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t"] Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121303 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e40c11-0e44-456d-b963-80aa609834b7" containerName="util" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121313 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e40c11-0e44-456d-b963-80aa609834b7" containerName="util" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121328 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e40c11-0e44-456d-b963-80aa609834b7" containerName="pull" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121333 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e40c11-0e44-456d-b963-80aa609834b7" containerName="pull" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121339 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22ab6509-1ed4-4cd0-b20c-c5307ab19494" containerName="console" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121345 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="22ab6509-1ed4-4cd0-b20c-c5307ab19494" containerName="console" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121352 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e40c11-0e44-456d-b963-80aa609834b7" containerName="extract" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121357 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e40c11-0e44-456d-b963-80aa609834b7" containerName="extract" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121409 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="22ab6509-1ed4-4cd0-b20c-c5307ab19494" containerName="console" Jul 03 20:44:26.121486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.121419 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e40c11-0e44-456d-b963-80aa609834b7" containerName="extract" Jul 03 20:44:26.125282 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.125265 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.141846 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.141825 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 03 20:44:26.141955 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.141876 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-52fnd\"" Jul 03 20:44:26.142575 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.142561 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:44:26.155314 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.155294 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t"] Jul 03 20:44:26.208387 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.208361 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrsg\" (UniqueName: \"kubernetes.io/projected/8dbb4174-01c2-4321-ac1a-c1ad85a01b42-kube-api-access-nvrsg\") pod \"cert-manager-operator-controller-manager-55c444bf86-rmf2t\" (UID: \"8dbb4174-01c2-4321-ac1a-c1ad85a01b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.208507 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.208391 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dbb4174-01c2-4321-ac1a-c1ad85a01b42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rmf2t\" (UID: \"8dbb4174-01c2-4321-ac1a-c1ad85a01b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.308959 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.308920 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dbb4174-01c2-4321-ac1a-c1ad85a01b42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rmf2t\" (UID: \"8dbb4174-01c2-4321-ac1a-c1ad85a01b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.309078 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.309012 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrsg\" (UniqueName: \"kubernetes.io/projected/8dbb4174-01c2-4321-ac1a-c1ad85a01b42-kube-api-access-nvrsg\") pod \"cert-manager-operator-controller-manager-55c444bf86-rmf2t\" (UID: \"8dbb4174-01c2-4321-ac1a-c1ad85a01b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.309296 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.309278 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dbb4174-01c2-4321-ac1a-c1ad85a01b42-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rmf2t\" (UID: \"8dbb4174-01c2-4321-ac1a-c1ad85a01b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.325231 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.325207 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrsg\" (UniqueName: \"kubernetes.io/projected/8dbb4174-01c2-4321-ac1a-c1ad85a01b42-kube-api-access-nvrsg\") pod \"cert-manager-operator-controller-manager-55c444bf86-rmf2t\" (UID: \"8dbb4174-01c2-4321-ac1a-c1ad85a01b42\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.433564 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.433503 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" Jul 03 20:44:26.562064 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.562028 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t"] Jul 03 20:44:26.566443 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:44:26.566419 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbb4174_01c2_4321_ac1a_c1ad85a01b42.slice/crio-641a19abfbcad9a5b35b34760718ea0367331133baa3ea59d205ff1b1192a888 WatchSource:0}: Error finding container 641a19abfbcad9a5b35b34760718ea0367331133baa3ea59d205ff1b1192a888: Status 404 returned error can't find the container with id 641a19abfbcad9a5b35b34760718ea0367331133baa3ea59d205ff1b1192a888 Jul 03 20:44:26.700424 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:26.700355 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" event={"ID":"8dbb4174-01c2-4321-ac1a-c1ad85a01b42","Type":"ContainerStarted","Data":"641a19abfbcad9a5b35b34760718ea0367331133baa3ea59d205ff1b1192a888"} Jul 03 20:44:29.715269 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:29.715232 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" event={"ID":"8dbb4174-01c2-4321-ac1a-c1ad85a01b42","Type":"ContainerStarted","Data":"60ec99d9a1db71d18a482ce0409a84ca2fec3093e9c3d764019d90a6d7ea3d3e"} Jul 03 20:44:29.746524 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:29.746477 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rmf2t" podStartSLOduration=0.786208999 podStartE2EDuration="3.746464241s" totalImagesPullingTime="2.960255242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:44:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:44:29.746257237 +0000 UTC m=+441.243824913" watchObservedRunningTime="2026-07-03 20:44:29.746464241 +0000 UTC m=+441.244031901" Jul 03 20:44:30.411184 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.411152 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz"] Jul 03 20:44:30.413477 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.413459 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.417634 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.417618 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-clrpt\"" Jul 03 20:44:30.418301 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.418289 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 20:44:30.421106 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.421084 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 20:44:30.433026 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.433006 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz"] Jul 03 20:44:30.539394 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.539368 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.539521 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.539415 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.539521 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.539441 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5vm8\" (UniqueName: \"kubernetes.io/projected/e4d1e88c-829d-474a-a3ed-76287fe6e431-kube-api-access-n5vm8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.640786 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.640730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.640941 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.640800 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.640941 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.640832 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5vm8\" (UniqueName: \"kubernetes.io/projected/e4d1e88c-829d-474a-a3ed-76287fe6e431-kube-api-access-n5vm8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.641215 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.641193 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.641250 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.641201 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.659049 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.659025 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5vm8\" (UniqueName: \"kubernetes.io/projected/e4d1e88c-829d-474a-a3ed-76287fe6e431-kube-api-access-n5vm8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.721994 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.721927 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:30.872235 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:30.872209 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz"] Jul 03 20:44:30.873717 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:44:30.873692 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4d1e88c_829d_474a_a3ed_76287fe6e431.slice/crio-2d1550219fa57b0bd3696779e0a615278d515cd3a0789918adf63ea2b70d664b WatchSource:0}: Error finding container 2d1550219fa57b0bd3696779e0a615278d515cd3a0789918adf63ea2b70d664b: Status 404 returned error can't find the container with id 2d1550219fa57b0bd3696779e0a615278d515cd3a0789918adf63ea2b70d664b Jul 03 20:44:31.724966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:31.724936 2558 generic.go:356] "Generic (PLEG): container finished" podID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerID="3f22683c4c1e470b8c2e4969ec80cc240db3411a50f51eebf8f1e92ca26ece2a" exitCode=0 Jul 03 20:44:31.725338 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:31.725020 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" event={"ID":"e4d1e88c-829d-474a-a3ed-76287fe6e431","Type":"ContainerDied","Data":"3f22683c4c1e470b8c2e4969ec80cc240db3411a50f51eebf8f1e92ca26ece2a"} Jul 03 20:44:31.725338 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:31.725071 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" event={"ID":"e4d1e88c-829d-474a-a3ed-76287fe6e431","Type":"ContainerStarted","Data":"2d1550219fa57b0bd3696779e0a615278d515cd3a0789918adf63ea2b70d664b"} Jul 03 20:44:33.585914 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.585879 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9"] Jul 03 20:44:33.592571 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.592551 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:33.595351 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.595331 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 03 20:44:33.595486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.595371 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 03 20:44:33.596469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.596421 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xdhjm\"" Jul 03 20:44:33.598605 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.598583 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9"] Jul 03 20:44:33.664633 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.664599 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5cd01f7-fa2e-494d-a731-bffdd2509641-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lq5t9\" (UID: \"b5cd01f7-fa2e-494d-a731-bffdd2509641\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:33.664781 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.664650 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljtrz\" (UniqueName: \"kubernetes.io/projected/b5cd01f7-fa2e-494d-a731-bffdd2509641-kube-api-access-ljtrz\") pod \"cert-manager-cainjector-559cdd8c87-lq5t9\" (UID: \"b5cd01f7-fa2e-494d-a731-bffdd2509641\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:33.765277 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.765245 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5cd01f7-fa2e-494d-a731-bffdd2509641-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lq5t9\" (UID: \"b5cd01f7-fa2e-494d-a731-bffdd2509641\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:33.765448 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.765290 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljtrz\" (UniqueName: \"kubernetes.io/projected/b5cd01f7-fa2e-494d-a731-bffdd2509641-kube-api-access-ljtrz\") pod \"cert-manager-cainjector-559cdd8c87-lq5t9\" (UID: \"b5cd01f7-fa2e-494d-a731-bffdd2509641\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:33.774367 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.774344 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljtrz\" (UniqueName: \"kubernetes.io/projected/b5cd01f7-fa2e-494d-a731-bffdd2509641-kube-api-access-ljtrz\") pod \"cert-manager-cainjector-559cdd8c87-lq5t9\" (UID: \"b5cd01f7-fa2e-494d-a731-bffdd2509641\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:33.774494 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.774382 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b5cd01f7-fa2e-494d-a731-bffdd2509641-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lq5t9\" (UID: \"b5cd01f7-fa2e-494d-a731-bffdd2509641\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:33.912019 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:33.911954 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" Jul 03 20:44:34.083411 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:34.083384 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9"] Jul 03 20:44:34.085555 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:44:34.085527 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5cd01f7_fa2e_494d_a731_bffdd2509641.slice/crio-8f2dc3bd1f1947c18cd9cdd014dfc3a1c5445621af80c347b1c2a89ed1b5e7f4 WatchSource:0}: Error finding container 8f2dc3bd1f1947c18cd9cdd014dfc3a1c5445621af80c347b1c2a89ed1b5e7f4: Status 404 returned error can't find the container with id 8f2dc3bd1f1947c18cd9cdd014dfc3a1c5445621af80c347b1c2a89ed1b5e7f4 Jul 03 20:44:34.741754 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:34.741714 2558 generic.go:356] "Generic (PLEG): container finished" podID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerID="03e9ead64f3ac8f9b0f70da94354210e0b6010672500d21876ef2ee9419715d9" exitCode=0 Jul 03 20:44:34.742205 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:34.741803 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" event={"ID":"e4d1e88c-829d-474a-a3ed-76287fe6e431","Type":"ContainerDied","Data":"03e9ead64f3ac8f9b0f70da94354210e0b6010672500d21876ef2ee9419715d9"} Jul 03 20:44:34.743059 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:34.743036 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" event={"ID":"b5cd01f7-fa2e-494d-a731-bffdd2509641","Type":"ContainerStarted","Data":"8f2dc3bd1f1947c18cd9cdd014dfc3a1c5445621af80c347b1c2a89ed1b5e7f4"} Jul 03 20:44:35.749125 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:35.749088 2558 generic.go:356] "Generic (PLEG): container finished" podID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerID="6190be75dce4cc1ac8c4af2c68fc6b7bcd2c81871fd5713b2961585b9ca5e896" exitCode=0 Jul 03 20:44:35.749588 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:35.749177 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" event={"ID":"e4d1e88c-829d-474a-a3ed-76287fe6e431","Type":"ContainerDied","Data":"6190be75dce4cc1ac8c4af2c68fc6b7bcd2c81871fd5713b2961585b9ca5e896"} Jul 03 20:44:37.669407 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.669383 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:37.759630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.759606 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" event={"ID":"e4d1e88c-829d-474a-a3ed-76287fe6e431","Type":"ContainerDied","Data":"2d1550219fa57b0bd3696779e0a615278d515cd3a0789918adf63ea2b70d664b"} Jul 03 20:44:37.759630 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.759628 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fptvcz" Jul 03 20:44:37.759776 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.759646 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d1550219fa57b0bd3696779e0a615278d515cd3a0789918adf63ea2b70d664b" Jul 03 20:44:37.797150 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.797124 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-bundle\") pod \"e4d1e88c-829d-474a-a3ed-76287fe6e431\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " Jul 03 20:44:37.797259 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.797179 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5vm8\" (UniqueName: \"kubernetes.io/projected/e4d1e88c-829d-474a-a3ed-76287fe6e431-kube-api-access-n5vm8\") pod \"e4d1e88c-829d-474a-a3ed-76287fe6e431\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " Jul 03 20:44:37.797259 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.797216 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-util\") pod \"e4d1e88c-829d-474a-a3ed-76287fe6e431\" (UID: \"e4d1e88c-829d-474a-a3ed-76287fe6e431\") " Jul 03 20:44:37.797571 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.797543 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-bundle" (OuterVolumeSpecName: "bundle") pod "e4d1e88c-829d-474a-a3ed-76287fe6e431" (UID: "e4d1e88c-829d-474a-a3ed-76287fe6e431"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:44:37.799166 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.799144 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4d1e88c-829d-474a-a3ed-76287fe6e431-kube-api-access-n5vm8" (OuterVolumeSpecName: "kube-api-access-n5vm8") pod "e4d1e88c-829d-474a-a3ed-76287fe6e431" (UID: "e4d1e88c-829d-474a-a3ed-76287fe6e431"). InnerVolumeSpecName "kube-api-access-n5vm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:44:37.801263 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.801242 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-util" (OuterVolumeSpecName: "util") pod "e4d1e88c-829d-474a-a3ed-76287fe6e431" (UID: "e4d1e88c-829d-474a-a3ed-76287fe6e431"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:44:37.898916 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.898840 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:37.898916 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.898876 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5vm8\" (UniqueName: \"kubernetes.io/projected/e4d1e88c-829d-474a-a3ed-76287fe6e431-kube-api-access-n5vm8\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:37.898916 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:37.898891 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e4d1e88c-829d-474a-a3ed-76287fe6e431-util\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:44:38.103356 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103329 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5qk9f"] Jul 03 20:44:38.103636 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103623 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerName="util" Jul 03 20:44:38.103636 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103637 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerName="util" Jul 03 20:44:38.103840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103650 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerName="extract" Jul 03 20:44:38.103840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103670 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerName="extract" Jul 03 20:44:38.103840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103679 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerName="pull" Jul 03 20:44:38.103840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103684 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerName="pull" Jul 03 20:44:38.103840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.103733 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4d1e88c-829d-474a-a3ed-76287fe6e431" containerName="extract" Jul 03 20:44:38.106658 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.106638 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.109591 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.109572 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sk247\"" Jul 03 20:44:38.119860 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.119839 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5qk9f"] Jul 03 20:44:38.201561 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.201533 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10b70e9f-8d47-4f9c-9758-a9c3cf077762-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5qk9f\" (UID: \"10b70e9f-8d47-4f9c-9758-a9c3cf077762\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.201722 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.201584 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khqzw\" (UniqueName: \"kubernetes.io/projected/10b70e9f-8d47-4f9c-9758-a9c3cf077762-kube-api-access-khqzw\") pod \"cert-manager-webhook-6486ffd997-5qk9f\" (UID: \"10b70e9f-8d47-4f9c-9758-a9c3cf077762\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.302850 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.302813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10b70e9f-8d47-4f9c-9758-a9c3cf077762-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5qk9f\" (UID: \"10b70e9f-8d47-4f9c-9758-a9c3cf077762\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.303013 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.302874 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khqzw\" (UniqueName: \"kubernetes.io/projected/10b70e9f-8d47-4f9c-9758-a9c3cf077762-kube-api-access-khqzw\") pod \"cert-manager-webhook-6486ffd997-5qk9f\" (UID: \"10b70e9f-8d47-4f9c-9758-a9c3cf077762\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.312636 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.312607 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10b70e9f-8d47-4f9c-9758-a9c3cf077762-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5qk9f\" (UID: \"10b70e9f-8d47-4f9c-9758-a9c3cf077762\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.312840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.312821 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khqzw\" (UniqueName: \"kubernetes.io/projected/10b70e9f-8d47-4f9c-9758-a9c3cf077762-kube-api-access-khqzw\") pod \"cert-manager-webhook-6486ffd997-5qk9f\" (UID: \"10b70e9f-8d47-4f9c-9758-a9c3cf077762\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.415884 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.415812 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.540721 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.540697 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5qk9f"] Jul 03 20:44:38.542381 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:44:38.542345 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10b70e9f_8d47_4f9c_9758_a9c3cf077762.slice/crio-135ffa37440c5a92fe225af5c21fe516b365188384c9fa26ac1c09bef01d8c76 WatchSource:0}: Error finding container 135ffa37440c5a92fe225af5c21fe516b365188384c9fa26ac1c09bef01d8c76: Status 404 returned error can't find the container with id 135ffa37440c5a92fe225af5c21fe516b365188384c9fa26ac1c09bef01d8c76 Jul 03 20:44:38.765665 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.765583 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" event={"ID":"b5cd01f7-fa2e-494d-a731-bffdd2509641","Type":"ContainerStarted","Data":"5c70508d045ffaa2f5e456aacc2b7bdb6793aec3b30ad6a7701ecb2a86d63898"} Jul 03 20:44:38.767165 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.767137 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" event={"ID":"10b70e9f-8d47-4f9c-9758-a9c3cf077762","Type":"ContainerStarted","Data":"d0db348d4d090a69f479a9f279e8871f271541ef7de2b12ea77f2d761f85a7a2"} Jul 03 20:44:38.767288 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.767167 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" event={"ID":"10b70e9f-8d47-4f9c-9758-a9c3cf077762","Type":"ContainerStarted","Data":"135ffa37440c5a92fe225af5c21fe516b365188384c9fa26ac1c09bef01d8c76"} Jul 03 20:44:38.767288 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.767184 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:38.782365 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.782320 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lq5t9" podStartSLOduration=2.155854425 podStartE2EDuration="5.782306633s" totalImagesPullingTime="3.626452208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:44:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:44:38.781096356 +0000 UTC m=+450.278664009" watchObservedRunningTime="2026-07-03 20:44:38.782306633 +0000 UTC m=+450.279874293" Jul 03 20:44:38.804441 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:38.804402 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" podStartSLOduration=0.80439023 podStartE2EDuration="804.39023ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:44:38.803427166 +0000 UTC m=+450.300994827" watchObservedRunningTime="2026-07-03 20:44:38.80439023 +0000 UTC m=+450.301957889" Jul 03 20:44:40.853065 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:40.853032 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nrs66"] Jul 03 20:44:40.856328 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:40.856308 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:40.858619 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:40.858604 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7zdww\"" Jul 03 20:44:40.877066 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:40.864032 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nrs66"] Jul 03 20:44:40.923299 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:40.923276 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/202d5c51-8e98-4b42-ada9-e3c6b9864aa7-bound-sa-token\") pod \"cert-manager-55b4d954d5-nrs66\" (UID: \"202d5c51-8e98-4b42-ada9-e3c6b9864aa7\") " pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:40.923409 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:40.923306 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drv4q\" (UniqueName: \"kubernetes.io/projected/202d5c51-8e98-4b42-ada9-e3c6b9864aa7-kube-api-access-drv4q\") pod \"cert-manager-55b4d954d5-nrs66\" (UID: \"202d5c51-8e98-4b42-ada9-e3c6b9864aa7\") " pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:41.024048 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.024016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/202d5c51-8e98-4b42-ada9-e3c6b9864aa7-bound-sa-token\") pod \"cert-manager-55b4d954d5-nrs66\" (UID: \"202d5c51-8e98-4b42-ada9-e3c6b9864aa7\") " pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:41.024048 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.024051 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drv4q\" (UniqueName: \"kubernetes.io/projected/202d5c51-8e98-4b42-ada9-e3c6b9864aa7-kube-api-access-drv4q\") pod \"cert-manager-55b4d954d5-nrs66\" (UID: \"202d5c51-8e98-4b42-ada9-e3c6b9864aa7\") " pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:41.032199 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.032169 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/202d5c51-8e98-4b42-ada9-e3c6b9864aa7-bound-sa-token\") pod \"cert-manager-55b4d954d5-nrs66\" (UID: \"202d5c51-8e98-4b42-ada9-e3c6b9864aa7\") " pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:41.032297 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.032285 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drv4q\" (UniqueName: \"kubernetes.io/projected/202d5c51-8e98-4b42-ada9-e3c6b9864aa7-kube-api-access-drv4q\") pod \"cert-manager-55b4d954d5-nrs66\" (UID: \"202d5c51-8e98-4b42-ada9-e3c6b9864aa7\") " pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:41.167448 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.167358 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nrs66" Jul 03 20:44:41.291959 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.291932 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nrs66"] Jul 03 20:44:41.293995 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:44:41.293965 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod202d5c51_8e98_4b42_ada9_e3c6b9864aa7.slice/crio-c25d24d6bfd3155cdac9b2efaadd55dfd2ad77468e93cffde2de1b8f0eba0d3e WatchSource:0}: Error finding container c25d24d6bfd3155cdac9b2efaadd55dfd2ad77468e93cffde2de1b8f0eba0d3e: Status 404 returned error can't find the container with id c25d24d6bfd3155cdac9b2efaadd55dfd2ad77468e93cffde2de1b8f0eba0d3e Jul 03 20:44:41.781692 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.781656 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nrs66" event={"ID":"202d5c51-8e98-4b42-ada9-e3c6b9864aa7","Type":"ContainerStarted","Data":"c30f99d9ab28026cd51469be65017773017f697ce7e492a5167a73c0c31833f0"} Jul 03 20:44:41.781692 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.781700 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nrs66" event={"ID":"202d5c51-8e98-4b42-ada9-e3c6b9864aa7","Type":"ContainerStarted","Data":"c25d24d6bfd3155cdac9b2efaadd55dfd2ad77468e93cffde2de1b8f0eba0d3e"} Jul 03 20:44:41.797890 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:41.797847 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nrs66" podStartSLOduration=1.797833228 podStartE2EDuration="1.797833228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:44:41.796706962 +0000 UTC m=+453.294274636" watchObservedRunningTime="2026-07-03 20:44:41.797833228 +0000 UTC m=+453.295400887" Jul 03 20:44:44.773563 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:44.773530 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5qk9f" Jul 03 20:44:58.524282 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.524250 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr"] Jul 03 20:44:58.527240 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.527225 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.529929 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.529906 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 03 20:44:58.530836 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.530821 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 03 20:44:58.530915 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.530830 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-clrpt\"" Jul 03 20:44:58.535735 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.535712 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr"] Jul 03 20:44:58.668728 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.668688 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8nqz\" (UniqueName: \"kubernetes.io/projected/8aeef542-1eb3-49f3-9376-ad05e4bb5328-kube-api-access-z8nqz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.668728 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.668733 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.668967 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.668754 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.770046 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.770016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8nqz\" (UniqueName: \"kubernetes.io/projected/8aeef542-1eb3-49f3-9376-ad05e4bb5328-kube-api-access-z8nqz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.770186 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.770055 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.770186 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.770085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.770504 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.770482 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.770540 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.770490 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.779466 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.779402 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8nqz\" (UniqueName: \"kubernetes.io/projected/8aeef542-1eb3-49f3-9376-ad05e4bb5328-kube-api-access-z8nqz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.837680 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.837651 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:44:58.961986 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:58.961962 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr"] Jul 03 20:44:58.963792 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:44:58.963758 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aeef542_1eb3_49f3_9376_ad05e4bb5328.slice/crio-6351e85f22e7e502912351c269d77e1cc7c979b61e7c6e06b3923017102e9854 WatchSource:0}: Error finding container 6351e85f22e7e502912351c269d77e1cc7c979b61e7c6e06b3923017102e9854: Status 404 returned error can't find the container with id 6351e85f22e7e502912351c269d77e1cc7c979b61e7c6e06b3923017102e9854 Jul 03 20:44:59.863696 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:59.863661 2558 generic.go:356] "Generic (PLEG): container finished" podID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerID="fc8b0a73d35ab6a5b0bfb9796068af5f85a0d30fa598dcd6df27ec07aa5573fb" exitCode=0 Jul 03 20:44:59.864133 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:59.863749 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" event={"ID":"8aeef542-1eb3-49f3-9376-ad05e4bb5328","Type":"ContainerDied","Data":"fc8b0a73d35ab6a5b0bfb9796068af5f85a0d30fa598dcd6df27ec07aa5573fb"} Jul 03 20:44:59.864133 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:44:59.863811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" event={"ID":"8aeef542-1eb3-49f3-9376-ad05e4bb5328","Type":"ContainerStarted","Data":"6351e85f22e7e502912351c269d77e1cc7c979b61e7c6e06b3923017102e9854"} Jul 03 20:45:00.870352 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:00.870315 2558 generic.go:356] "Generic (PLEG): container finished" podID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerID="406debb2ba10f76480c7e7311ccf44d432e747de128b6d155d84d60b61f5eb2a" exitCode=0 Jul 03 20:45:00.870813 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:00.870398 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" event={"ID":"8aeef542-1eb3-49f3-9376-ad05e4bb5328","Type":"ContainerDied","Data":"406debb2ba10f76480c7e7311ccf44d432e747de128b6d155d84d60b61f5eb2a"} Jul 03 20:45:01.877185 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:01.877154 2558 generic.go:356] "Generic (PLEG): container finished" podID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerID="eee381e79ee3336a4839d8213362aca702cd71665caa00fe8fc796ea86e83658" exitCode=0 Jul 03 20:45:01.877573 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:01.877222 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" event={"ID":"8aeef542-1eb3-49f3-9376-ad05e4bb5328","Type":"ContainerDied","Data":"eee381e79ee3336a4839d8213362aca702cd71665caa00fe8fc796ea86e83658"} Jul 03 20:45:03.008523 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.008494 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:45:03.106507 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.106474 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-util\") pod \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " Jul 03 20:45:03.106688 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.106540 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8nqz\" (UniqueName: \"kubernetes.io/projected/8aeef542-1eb3-49f3-9376-ad05e4bb5328-kube-api-access-z8nqz\") pod \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " Jul 03 20:45:03.106688 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.106558 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-bundle\") pod \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\" (UID: \"8aeef542-1eb3-49f3-9376-ad05e4bb5328\") " Jul 03 20:45:03.107284 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.107256 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-bundle" (OuterVolumeSpecName: "bundle") pod "8aeef542-1eb3-49f3-9376-ad05e4bb5328" (UID: "8aeef542-1eb3-49f3-9376-ad05e4bb5328"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:45:03.108574 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.108547 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aeef542-1eb3-49f3-9376-ad05e4bb5328-kube-api-access-z8nqz" (OuterVolumeSpecName: "kube-api-access-z8nqz") pod "8aeef542-1eb3-49f3-9376-ad05e4bb5328" (UID: "8aeef542-1eb3-49f3-9376-ad05e4bb5328"). InnerVolumeSpecName "kube-api-access-z8nqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:45:03.112335 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.112300 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-util" (OuterVolumeSpecName: "util") pod "8aeef542-1eb3-49f3-9376-ad05e4bb5328" (UID: "8aeef542-1eb3-49f3-9376-ad05e4bb5328"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:45:03.208051 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.208012 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8nqz\" (UniqueName: \"kubernetes.io/projected/8aeef542-1eb3-49f3-9376-ad05e4bb5328-kube-api-access-z8nqz\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:45:03.208051 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.208044 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:45:03.208051 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.208058 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8aeef542-1eb3-49f3-9376-ad05e4bb5328-util\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:45:03.888423 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.888393 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" event={"ID":"8aeef542-1eb3-49f3-9376-ad05e4bb5328","Type":"ContainerDied","Data":"6351e85f22e7e502912351c269d77e1cc7c979b61e7c6e06b3923017102e9854"} Jul 03 20:45:03.888585 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.888441 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6351e85f22e7e502912351c269d77e1cc7c979b61e7c6e06b3923017102e9854" Jul 03 20:45:03.888585 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:03.888403 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affkhzr" Jul 03 20:45:10.729441 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729410 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z"] Jul 03 20:45:10.729843 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729729 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerName="extract" Jul 03 20:45:10.729843 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729739 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerName="extract" Jul 03 20:45:10.729843 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729749 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerName="util" Jul 03 20:45:10.729843 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729754 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerName="util" Jul 03 20:45:10.729843 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729777 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerName="pull" Jul 03 20:45:10.729843 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729783 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerName="pull" Jul 03 20:45:10.730033 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.729872 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aeef542-1eb3-49f3-9376-ad05e4bb5328" containerName="extract" Jul 03 20:45:10.734458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.734428 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.736995 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.736965 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 03 20:45:10.737106 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.736974 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 03 20:45:10.737234 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.737211 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8vxtf\"" Jul 03 20:45:10.737337 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.737254 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 03 20:45:10.737337 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.737297 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 03 20:45:10.748977 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.748949 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z"] Jul 03 20:45:10.770808 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.770748 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11e72f28-02b2-4452-9c99-0db7996119cb-tmp\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.771090 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.770873 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11e72f28-02b2-4452-9c99-0db7996119cb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.771090 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.770901 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcszn\" (UniqueName: \"kubernetes.io/projected/11e72f28-02b2-4452-9c99-0db7996119cb-kube-api-access-lcszn\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.771090 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.770942 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/11e72f28-02b2-4452-9c99-0db7996119cb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.771090 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.771027 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/11e72f28-02b2-4452-9c99-0db7996119cb-manifests\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.871396 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.871363 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11e72f28-02b2-4452-9c99-0db7996119cb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.871396 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.871398 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcszn\" (UniqueName: \"kubernetes.io/projected/11e72f28-02b2-4452-9c99-0db7996119cb-kube-api-access-lcszn\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.871632 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.871428 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/11e72f28-02b2-4452-9c99-0db7996119cb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.871632 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.871475 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/11e72f28-02b2-4452-9c99-0db7996119cb-manifests\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.871632 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.871537 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11e72f28-02b2-4452-9c99-0db7996119cb-tmp\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.872026 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.871992 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/11e72f28-02b2-4452-9c99-0db7996119cb-manifests\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.873745 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.873718 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11e72f28-02b2-4452-9c99-0db7996119cb-tmp\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.873987 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.873967 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11e72f28-02b2-4452-9c99-0db7996119cb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.874101 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.874083 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/11e72f28-02b2-4452-9c99-0db7996119cb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:10.880053 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:10.880030 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcszn\" (UniqueName: \"kubernetes.io/projected/11e72f28-02b2-4452-9c99-0db7996119cb-kube-api-access-lcszn\") pod \"opendatahub-operator-controller-manager-5448f4d7cf-kd78z\" (UID: \"11e72f28-02b2-4452-9c99-0db7996119cb\") " pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:11.044815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:11.044789 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:11.193284 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:11.193255 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z"] Jul 03 20:45:11.196400 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:45:11.196373 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11e72f28_02b2_4452_9c99_0db7996119cb.slice/crio-e6fe3baff06d8161ed4c5717165141d569a16ae2cf275b21d3366d99edd5c58d WatchSource:0}: Error finding container e6fe3baff06d8161ed4c5717165141d569a16ae2cf275b21d3366d99edd5c58d: Status 404 returned error can't find the container with id e6fe3baff06d8161ed4c5717165141d569a16ae2cf275b21d3366d99edd5c58d Jul 03 20:45:11.928242 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:11.928202 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" event={"ID":"11e72f28-02b2-4452-9c99-0db7996119cb","Type":"ContainerStarted","Data":"e6fe3baff06d8161ed4c5717165141d569a16ae2cf275b21d3366d99edd5c58d"} Jul 03 20:45:14.942831 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:14.942798 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" event={"ID":"11e72f28-02b2-4452-9c99-0db7996119cb","Type":"ContainerStarted","Data":"4d3c1c56fc9ce49505d52bc64062d6e1bffd26a7006f3677d5d79f644231f087"} Jul 03 20:45:15.948966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:15.948937 2558 generic.go:356] "Generic (PLEG): container finished" podID="11e72f28-02b2-4452-9c99-0db7996119cb" containerID="4d3c1c56fc9ce49505d52bc64062d6e1bffd26a7006f3677d5d79f644231f087" exitCode=0 Jul 03 20:45:15.949331 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:15.949009 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" event={"ID":"11e72f28-02b2-4452-9c99-0db7996119cb","Type":"ContainerDied","Data":"4d3c1c56fc9ce49505d52bc64062d6e1bffd26a7006f3677d5d79f644231f087"} Jul 03 20:45:16.955690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:16.955648 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" event={"ID":"11e72f28-02b2-4452-9c99-0db7996119cb","Type":"ContainerStarted","Data":"fa72e745e982c08c098d47e16b96d10862221b295f39314f23f9d5058abd18e4"} Jul 03 20:45:16.956119 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:16.955802 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:16.976806 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:16.976739 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" podStartSLOduration=2.649794159 podStartE2EDuration="6.976726022s" totalImagesPullingTime="3.152738401s" totalInitContainerRuntime="1.174193462s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:45:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:45:16.976543794 +0000 UTC m=+488.474111467" watchObservedRunningTime="2026-07-03 20:45:16.976726022 +0000 UTC m=+488.474293681" Jul 03 20:45:27.962563 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:27.962536 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5448f4d7cf-kd78z" Jul 03 20:45:28.417150 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.417114 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78"] Jul 03 20:45:28.429715 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.429682 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.429944 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.429918 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78"] Jul 03 20:45:28.433687 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.433667 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 03 20:45:28.433801 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.433667 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 03 20:45:28.433875 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.433667 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 03 20:45:28.433950 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.433933 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 03 20:45:28.434013 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.433948 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-shghf\"" Jul 03 20:45:28.434068 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.433937 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 03 20:45:28.496957 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.496923 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcq47\" (UniqueName: \"kubernetes.io/projected/dca714b8-d854-4852-a8d6-60383ac671cb-kube-api-access-xcq47\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.496957 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.496957 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dca714b8-d854-4852-a8d6-60383ac671cb-cert\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.497149 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.496980 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dca714b8-d854-4852-a8d6-60383ac671cb-metrics-cert\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.497149 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.497056 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dca714b8-d854-4852-a8d6-60383ac671cb-manager-config\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.598216 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.598187 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcq47\" (UniqueName: \"kubernetes.io/projected/dca714b8-d854-4852-a8d6-60383ac671cb-kube-api-access-xcq47\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.598216 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.598217 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dca714b8-d854-4852-a8d6-60383ac671cb-cert\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.598394 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.598325 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dca714b8-d854-4852-a8d6-60383ac671cb-metrics-cert\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.598394 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.598386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dca714b8-d854-4852-a8d6-60383ac671cb-manager-config\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.599042 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.599024 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dca714b8-d854-4852-a8d6-60383ac671cb-manager-config\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.600673 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.600639 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dca714b8-d854-4852-a8d6-60383ac671cb-cert\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.600802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.600721 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dca714b8-d854-4852-a8d6-60383ac671cb-metrics-cert\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.606283 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.606262 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcq47\" (UniqueName: \"kubernetes.io/projected/dca714b8-d854-4852-a8d6-60383ac671cb-kube-api-access-xcq47\") pod \"lws-controller-manager-66c99c87c9-knn78\" (UID: \"dca714b8-d854-4852-a8d6-60383ac671cb\") " pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.739459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.739377 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:28.868090 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:28.868063 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78"] Jul 03 20:45:28.869699 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:45:28.869665 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca714b8_d854_4852_a8d6_60383ac671cb.slice/crio-2ca8bca0930793964ad5194315f6b12b00efb428b487cff34dbcb3f7f9c79f6c WatchSource:0}: Error finding container 2ca8bca0930793964ad5194315f6b12b00efb428b487cff34dbcb3f7f9c79f6c: Status 404 returned error can't find the container with id 2ca8bca0930793964ad5194315f6b12b00efb428b487cff34dbcb3f7f9c79f6c Jul 03 20:45:29.013459 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:29.013380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" event={"ID":"dca714b8-d854-4852-a8d6-60383ac671cb","Type":"ContainerStarted","Data":"2ca8bca0930793964ad5194315f6b12b00efb428b487cff34dbcb3f7f9c79f6c"} Jul 03 20:45:31.024146 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:31.024112 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" event={"ID":"dca714b8-d854-4852-a8d6-60383ac671cb","Type":"ContainerStarted","Data":"7781db7b9d3b3144eda160731ef5b621b3f3a5b700e508014b01dd6d172cb203"} Jul 03 20:45:31.024628 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:31.024236 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:31.043077 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:31.043027 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" podStartSLOduration=1.6773921299999999 podStartE2EDuration="3.043010887s" totalImagesPullingTime="1.365618757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:45:31.041077963 +0000 UTC m=+502.538645624" watchObservedRunningTime="2026-07-03 20:45:31.043010887 +0000 UTC m=+502.540578571" Jul 03 20:45:38.407495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.407461 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz"] Jul 03 20:45:38.410725 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.410710 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.413326 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.413302 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 03 20:45:38.413444 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.413426 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 03 20:45:38.413513 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.413464 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2lkfv\"" Jul 03 20:45:38.413568 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.413541 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 03 20:45:38.413705 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.413689 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 03 20:45:38.413783 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.413720 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 03 20:45:38.413783 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.413722 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 03 20:45:38.421248 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.421225 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz"] Jul 03 20:45:38.487610 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.487582 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.487821 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.487642 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.487821 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.487723 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8cff6673-c65f-43b9-aa62-992749052eaf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.487821 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.487775 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cff6673-c65f-43b9-aa62-992749052eaf-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.487821 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.487795 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzzmt\" (UniqueName: \"kubernetes.io/projected/8cff6673-c65f-43b9-aa62-992749052eaf-kube-api-access-rzzmt\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.488003 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.487833 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cff6673-c65f-43b9-aa62-992749052eaf-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.488003 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.487859 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.588379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.588379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588388 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8cff6673-c65f-43b9-aa62-992749052eaf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.588610 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588536 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cff6673-c65f-43b9-aa62-992749052eaf-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.588610 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588583 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzzmt\" (UniqueName: \"kubernetes.io/projected/8cff6673-c65f-43b9-aa62-992749052eaf-kube-api-access-rzzmt\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.588725 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588615 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cff6673-c65f-43b9-aa62-992749052eaf-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.588725 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.588725 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.589024 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.588993 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8cff6673-c65f-43b9-aa62-992749052eaf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.590814 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.590752 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cff6673-c65f-43b9-aa62-992749052eaf-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.590932 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.590851 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.590975 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.590940 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.591060 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.591041 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8cff6673-c65f-43b9-aa62-992749052eaf-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.596568 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.596543 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cff6673-c65f-43b9-aa62-992749052eaf-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.596870 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.596852 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzzmt\" (UniqueName: \"kubernetes.io/projected/8cff6673-c65f-43b9-aa62-992749052eaf-kube-api-access-rzzmt\") pod \"istiod-openshift-gateway-94bb8fbfd-qsftz\" (UID: \"8cff6673-c65f-43b9-aa62-992749052eaf\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.720805 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.720708 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:38.886352 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:38.886325 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz"] Jul 03 20:45:38.887237 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:45:38.887215 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cff6673_c65f_43b9_aa62_992749052eaf.slice/crio-ad064f9d42c6cbef387ba2d4ec3de115950bbedb8eefa673165639f831068e44 WatchSource:0}: Error finding container ad064f9d42c6cbef387ba2d4ec3de115950bbedb8eefa673165639f831068e44: Status 404 returned error can't find the container with id ad064f9d42c6cbef387ba2d4ec3de115950bbedb8eefa673165639f831068e44 Jul 03 20:45:39.069513 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:39.069464 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" event={"ID":"8cff6673-c65f-43b9-aa62-992749052eaf","Type":"ContainerStarted","Data":"ad064f9d42c6cbef387ba2d4ec3de115950bbedb8eefa673165639f831068e44"} Jul 03 20:45:41.152613 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:41.152566 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 20:45:41.153029 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:41.152658 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 20:45:42.031824 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:42.031795 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66c99c87c9-knn78" Jul 03 20:45:42.091358 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:42.091327 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" event={"ID":"8cff6673-c65f-43b9-aa62-992749052eaf","Type":"ContainerStarted","Data":"b39ab2052e0bc7a9d9787489db7203295fd9d1f787b0693caf9be4d2a5b1ba6e"} Jul 03 20:45:42.091685 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:42.091634 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:45:42.093080 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:42.093047 2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qsftz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 03 20:45:42.093217 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:42.093100 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" podUID="8cff6673-c65f-43b9-aa62-992749052eaf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 20:45:42.120949 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:42.120902 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" podStartSLOduration=1.85778537 podStartE2EDuration="4.120886628s" totalImagesPullingTime="2.263101258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:45:42.118596661 +0000 UTC m=+513.616164343" watchObservedRunningTime="2026-07-03 20:45:42.120886628 +0000 UTC m=+513.618454289" Jul 03 20:45:43.096373 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:45:43.096345 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qsftz" Jul 03 20:46:46.008206 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.008172 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj"] Jul 03 20:46:46.011575 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.011559 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.014122 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.014099 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c9dnh\"" Jul 03 20:46:46.014213 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.014109 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 20:46:46.015042 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.015028 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 20:46:46.019457 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.019435 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj"] Jul 03 20:46:46.174032 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.173996 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.174222 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.174127 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tssrd\" (UniqueName: \"kubernetes.io/projected/7c6c46d6-2abe-4693-ae94-608e27c0c2af-kube-api-access-tssrd\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.174222 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.174166 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.274908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.274833 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.274908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.274904 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tssrd\" (UniqueName: \"kubernetes.io/projected/7c6c46d6-2abe-4693-ae94-608e27c0c2af-kube-api-access-tssrd\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.275072 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.274928 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.275288 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.275264 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.275328 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.275275 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.283301 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.283275 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tssrd\" (UniqueName: \"kubernetes.io/projected/7c6c46d6-2abe-4693-ae94-608e27c0c2af-kube-api-access-tssrd\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.321109 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.321077 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:46.451961 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.451929 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj"] Jul 03 20:46:46.455033 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:46:46.454997 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6c46d6_2abe_4693_ae94_608e27c0c2af.slice/crio-cf122731aa4a21a1bb78136f4de2e7709948a668e24b8cb5605d34586eeac8e7 WatchSource:0}: Error finding container cf122731aa4a21a1bb78136f4de2e7709948a668e24b8cb5605d34586eeac8e7: Status 404 returned error can't find the container with id cf122731aa4a21a1bb78136f4de2e7709948a668e24b8cb5605d34586eeac8e7 Jul 03 20:46:46.814828 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.814731 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c"] Jul 03 20:46:46.818278 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.818264 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:46.826949 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.826925 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c"] Jul 03 20:46:46.981712 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.981674 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:46.981902 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.981729 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:46.981902 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:46.981809 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjlp\" (UniqueName: \"kubernetes.io/projected/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-kube-api-access-dsjlp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.082629 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.082553 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.082629 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.082598 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.082629 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.082618 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjlp\" (UniqueName: \"kubernetes.io/projected/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-kube-api-access-dsjlp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.083069 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.083019 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.083069 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.083030 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.092886 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.092861 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjlp\" (UniqueName: \"kubernetes.io/projected/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-kube-api-access-dsjlp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.127678 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.127650 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:47.255028 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.254990 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c"] Jul 03 20:46:47.255800 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:46:47.255749 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d7c9fbf_1a21_4f60_840f_ab563bda1e98.slice/crio-7fe02ac4195cee0ced444abd4ef25aab7ba3447875d29445fc01bffb6f2d8529 WatchSource:0}: Error finding container 7fe02ac4195cee0ced444abd4ef25aab7ba3447875d29445fc01bffb6f2d8529: Status 404 returned error can't find the container with id 7fe02ac4195cee0ced444abd4ef25aab7ba3447875d29445fc01bffb6f2d8529 Jul 03 20:46:47.391090 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.391055 2558 generic.go:356] "Generic (PLEG): container finished" podID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerID="f8f6556d778188351420400d01f163402eb472d02f288004fc9c2cd625e8a775" exitCode=0 Jul 03 20:46:47.391239 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.391139 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" event={"ID":"7c6c46d6-2abe-4693-ae94-608e27c0c2af","Type":"ContainerDied","Data":"f8f6556d778188351420400d01f163402eb472d02f288004fc9c2cd625e8a775"} Jul 03 20:46:47.391239 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.391191 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" event={"ID":"7c6c46d6-2abe-4693-ae94-608e27c0c2af","Type":"ContainerStarted","Data":"cf122731aa4a21a1bb78136f4de2e7709948a668e24b8cb5605d34586eeac8e7"} Jul 03 20:46:47.392647 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.392626 2558 generic.go:356] "Generic (PLEG): container finished" podID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerID="65ffd4de837bdb1c2d91a982fb6abaf22d18041ae68471c42e82ae4c6c073b43" exitCode=0 Jul 03 20:46:47.392740 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.392679 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" event={"ID":"9d7c9fbf-1a21-4f60-840f-ab563bda1e98","Type":"ContainerDied","Data":"65ffd4de837bdb1c2d91a982fb6abaf22d18041ae68471c42e82ae4c6c073b43"} Jul 03 20:46:47.392740 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.392715 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" event={"ID":"9d7c9fbf-1a21-4f60-840f-ab563bda1e98","Type":"ContainerStarted","Data":"7fe02ac4195cee0ced444abd4ef25aab7ba3447875d29445fc01bffb6f2d8529"} Jul 03 20:46:47.410923 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.410899 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56"] Jul 03 20:46:47.415031 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.415010 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.422665 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.422646 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56"] Jul 03 20:46:47.586915 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.586885 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.587080 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.586933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.587080 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.586959 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdkhr\" (UniqueName: \"kubernetes.io/projected/2b737fc0-5a2c-448c-95cf-1e86361c3610-kube-api-access-sdkhr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.688241 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.688164 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.688241 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.688210 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.688241 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.688239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdkhr\" (UniqueName: \"kubernetes.io/projected/2b737fc0-5a2c-448c-95cf-1e86361c3610-kube-api-access-sdkhr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.688547 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.688526 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.688603 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.688587 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.698090 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.698066 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdkhr\" (UniqueName: \"kubernetes.io/projected/2b737fc0-5a2c-448c-95cf-1e86361c3610-kube-api-access-sdkhr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.738439 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.738413 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:47.886425 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.886389 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm"] Jul 03 20:46:47.891023 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:46:47.890997 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b737fc0_5a2c_448c_95cf_1e86361c3610.slice/crio-6ba0151ed3d320c7f821624d862b31751ab4697c499809d010a0da7952190381 WatchSource:0}: Error finding container 6ba0151ed3d320c7f821624d862b31751ab4697c499809d010a0da7952190381: Status 404 returned error can't find the container with id 6ba0151ed3d320c7f821624d862b31751ab4697c499809d010a0da7952190381 Jul 03 20:46:47.891344 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.891327 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56"] Jul 03 20:46:47.891431 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.891421 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:47.912100 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:47.912074 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm"] Jul 03 20:46:48.091397 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.091363 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.091802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.091403 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgkd2\" (UniqueName: \"kubernetes.io/projected/47890d2f-69ac-4d41-9dd7-1f755a28414e-kube-api-access-zgkd2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.091802 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.091494 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.193131 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.193058 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.193709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.193356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.193709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.193408 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgkd2\" (UniqueName: \"kubernetes.io/projected/47890d2f-69ac-4d41-9dd7-1f755a28414e-kube-api-access-zgkd2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.193709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.193450 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.193709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.193642 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.202265 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.202238 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgkd2\" (UniqueName: \"kubernetes.io/projected/47890d2f-69ac-4d41-9dd7-1f755a28414e-kube-api-access-zgkd2\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.204083 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.204016 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:48.335828 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.335737 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm"] Jul 03 20:46:48.338050 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:46:48.338010 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47890d2f_69ac_4d41_9dd7_1f755a28414e.slice/crio-94028bacd91a6ef1d55257a1c3f63b3dc71388cbcf61ab4a86bc0820b5967e9f WatchSource:0}: Error finding container 94028bacd91a6ef1d55257a1c3f63b3dc71388cbcf61ab4a86bc0820b5967e9f: Status 404 returned error can't find the container with id 94028bacd91a6ef1d55257a1c3f63b3dc71388cbcf61ab4a86bc0820b5967e9f Jul 03 20:46:48.400555 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.400533 2558 generic.go:356] "Generic (PLEG): container finished" podID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerID="cea3652a8380f38f69cf1792e1e601297a739868a83c0530b624dc80eb1ab2a6" exitCode=0 Jul 03 20:46:48.400666 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.400594 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" event={"ID":"7c6c46d6-2abe-4693-ae94-608e27c0c2af","Type":"ContainerDied","Data":"cea3652a8380f38f69cf1792e1e601297a739868a83c0530b624dc80eb1ab2a6"} Jul 03 20:46:48.402317 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.402084 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" event={"ID":"47890d2f-69ac-4d41-9dd7-1f755a28414e","Type":"ContainerStarted","Data":"94028bacd91a6ef1d55257a1c3f63b3dc71388cbcf61ab4a86bc0820b5967e9f"} Jul 03 20:46:48.403987 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.403963 2558 generic.go:356] "Generic (PLEG): container finished" podID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerID="5264252384a6625eca0611f2db2aa1031ca2c192594185c6f2085d1fdfd57a92" exitCode=0 Jul 03 20:46:48.404077 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.404037 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" event={"ID":"9d7c9fbf-1a21-4f60-840f-ab563bda1e98","Type":"ContainerDied","Data":"5264252384a6625eca0611f2db2aa1031ca2c192594185c6f2085d1fdfd57a92"} Jul 03 20:46:48.405663 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.405553 2558 generic.go:356] "Generic (PLEG): container finished" podID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerID="c8e976b6aa5507d76c337a983c20df739f74af4b15e85756fc4fded180e528b6" exitCode=0 Jul 03 20:46:48.405663 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.405622 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" event={"ID":"2b737fc0-5a2c-448c-95cf-1e86361c3610","Type":"ContainerDied","Data":"c8e976b6aa5507d76c337a983c20df739f74af4b15e85756fc4fded180e528b6"} Jul 03 20:46:48.405816 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:48.405664 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" event={"ID":"2b737fc0-5a2c-448c-95cf-1e86361c3610","Type":"ContainerStarted","Data":"6ba0151ed3d320c7f821624d862b31751ab4697c499809d010a0da7952190381"} Jul 03 20:46:49.416169 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.416141 2558 generic.go:356] "Generic (PLEG): container finished" podID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerID="dc9397d5702e4b2cb14ab32c23491b21af87dace74f7be23868248a863af53ba" exitCode=0 Jul 03 20:46:49.416515 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.416202 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" event={"ID":"7c6c46d6-2abe-4693-ae94-608e27c0c2af","Type":"ContainerDied","Data":"dc9397d5702e4b2cb14ab32c23491b21af87dace74f7be23868248a863af53ba"} Jul 03 20:46:49.417568 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.417542 2558 generic.go:356] "Generic (PLEG): container finished" podID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerID="cdcf8813e3345e1b4420d5c1f7781b05b723ad0cb7888e949cc9dccd05a5422c" exitCode=0 Jul 03 20:46:49.417679 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.417624 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" event={"ID":"47890d2f-69ac-4d41-9dd7-1f755a28414e","Type":"ContainerDied","Data":"cdcf8813e3345e1b4420d5c1f7781b05b723ad0cb7888e949cc9dccd05a5422c"} Jul 03 20:46:49.419921 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.419842 2558 generic.go:356] "Generic (PLEG): container finished" podID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerID="cb98a0dbfef8965fce90d5ba9265e3d6c9a6110555385519da359af38f395a37" exitCode=0 Jul 03 20:46:49.419993 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.419918 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" event={"ID":"9d7c9fbf-1a21-4f60-840f-ab563bda1e98","Type":"ContainerDied","Data":"cb98a0dbfef8965fce90d5ba9265e3d6c9a6110555385519da359af38f395a37"} Jul 03 20:46:49.421713 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.421694 2558 generic.go:356] "Generic (PLEG): container finished" podID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerID="40540216b4b2d11ccd34a86c7ff4ac5af5b524f53c9e79486c3275e831f004ef" exitCode=0 Jul 03 20:46:49.421814 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:49.421796 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" event={"ID":"2b737fc0-5a2c-448c-95cf-1e86361c3610","Type":"ContainerDied","Data":"40540216b4b2d11ccd34a86c7ff4ac5af5b524f53c9e79486c3275e831f004ef"} Jul 03 20:46:50.434783 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.434737 2558 generic.go:356] "Generic (PLEG): container finished" podID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerID="bb2248410526d4ff09e26d7236703f6c5e0bb2f88d5dd2860e6c5601da15c63d" exitCode=0 Jul 03 20:46:50.435251 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.434819 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" event={"ID":"47890d2f-69ac-4d41-9dd7-1f755a28414e","Type":"ContainerDied","Data":"bb2248410526d4ff09e26d7236703f6c5e0bb2f88d5dd2860e6c5601da15c63d"} Jul 03 20:46:50.437374 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.437352 2558 generic.go:356] "Generic (PLEG): container finished" podID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerID="3d257e44b82eebba3ad550903980737185ac6698836519bc89d1a903829d83ac" exitCode=0 Jul 03 20:46:50.437886 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.437500 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" event={"ID":"2b737fc0-5a2c-448c-95cf-1e86361c3610","Type":"ContainerDied","Data":"3d257e44b82eebba3ad550903980737185ac6698836519bc89d1a903829d83ac"} Jul 03 20:46:50.653709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.653686 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:50.656913 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.656897 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:50.714528 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.714504 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-bundle\") pod \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " Jul 03 20:46:50.714631 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.714560 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tssrd\" (UniqueName: \"kubernetes.io/projected/7c6c46d6-2abe-4693-ae94-608e27c0c2af-kube-api-access-tssrd\") pod \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " Jul 03 20:46:50.714631 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.714603 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-util\") pod \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\" (UID: \"7c6c46d6-2abe-4693-ae94-608e27c0c2af\") " Jul 03 20:46:50.714631 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.714627 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsjlp\" (UniqueName: \"kubernetes.io/projected/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-kube-api-access-dsjlp\") pod \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " Jul 03 20:46:50.714805 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.714696 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-util\") pod \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " Jul 03 20:46:50.714805 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.714740 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-bundle\") pod \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\" (UID: \"9d7c9fbf-1a21-4f60-840f-ab563bda1e98\") " Jul 03 20:46:50.715486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.715391 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-bundle" (OuterVolumeSpecName: "bundle") pod "7c6c46d6-2abe-4693-ae94-608e27c0c2af" (UID: "7c6c46d6-2abe-4693-ae94-608e27c0c2af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:50.715486 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.715401 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-bundle" (OuterVolumeSpecName: "bundle") pod "9d7c9fbf-1a21-4f60-840f-ab563bda1e98" (UID: "9d7c9fbf-1a21-4f60-840f-ab563bda1e98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:50.717468 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.717440 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-kube-api-access-dsjlp" (OuterVolumeSpecName: "kube-api-access-dsjlp") pod "9d7c9fbf-1a21-4f60-840f-ab563bda1e98" (UID: "9d7c9fbf-1a21-4f60-840f-ab563bda1e98"). InnerVolumeSpecName "kube-api-access-dsjlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:46:50.717575 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.717496 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6c46d6-2abe-4693-ae94-608e27c0c2af-kube-api-access-tssrd" (OuterVolumeSpecName: "kube-api-access-tssrd") pod "7c6c46d6-2abe-4693-ae94-608e27c0c2af" (UID: "7c6c46d6-2abe-4693-ae94-608e27c0c2af"). InnerVolumeSpecName "kube-api-access-tssrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:46:50.720283 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.720262 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-util" (OuterVolumeSpecName: "util") pod "9d7c9fbf-1a21-4f60-840f-ab563bda1e98" (UID: "9d7c9fbf-1a21-4f60-840f-ab563bda1e98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:50.721101 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.721068 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-util" (OuterVolumeSpecName: "util") pod "7c6c46d6-2abe-4693-ae94-608e27c0c2af" (UID: "7c6c46d6-2abe-4693-ae94-608e27c0c2af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:50.816171 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.816145 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-util\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:50.816275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.816172 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsjlp\" (UniqueName: \"kubernetes.io/projected/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-kube-api-access-dsjlp\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:50.816275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.816187 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-util\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:50.816275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.816199 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d7c9fbf-1a21-4f60-840f-ab563bda1e98-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:50.816275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.816211 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c6c46d6-2abe-4693-ae94-608e27c0c2af-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:50.816275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:50.816222 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tssrd\" (UniqueName: \"kubernetes.io/projected/7c6c46d6-2abe-4693-ae94-608e27c0c2af-kube-api-access-tssrd\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:51.444775 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.444740 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" event={"ID":"7c6c46d6-2abe-4693-ae94-608e27c0c2af","Type":"ContainerDied","Data":"cf122731aa4a21a1bb78136f4de2e7709948a668e24b8cb5605d34586eeac8e7"} Jul 03 20:46:51.445208 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.444812 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf122731aa4a21a1bb78136f4de2e7709948a668e24b8cb5605d34586eeac8e7" Jul 03 20:46:51.445208 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.444752 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj" Jul 03 20:46:51.447069 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.447044 2558 generic.go:356] "Generic (PLEG): container finished" podID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerID="d7d09eb062793d8518d1b9cf24e5201415189b1d288b82517f9917470c6ce4b1" exitCode=0 Jul 03 20:46:51.447205 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.447080 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" event={"ID":"47890d2f-69ac-4d41-9dd7-1f755a28414e","Type":"ContainerDied","Data":"d7d09eb062793d8518d1b9cf24e5201415189b1d288b82517f9917470c6ce4b1"} Jul 03 20:46:51.448795 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.448777 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" Jul 03 20:46:51.448885 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.448803 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c" event={"ID":"9d7c9fbf-1a21-4f60-840f-ab563bda1e98","Type":"ContainerDied","Data":"7fe02ac4195cee0ced444abd4ef25aab7ba3447875d29445fc01bffb6f2d8529"} Jul 03 20:46:51.448885 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.448832 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fe02ac4195cee0ced444abd4ef25aab7ba3447875d29445fc01bffb6f2d8529" Jul 03 20:46:51.583201 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.583176 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:51.622868 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.622848 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdkhr\" (UniqueName: \"kubernetes.io/projected/2b737fc0-5a2c-448c-95cf-1e86361c3610-kube-api-access-sdkhr\") pod \"2b737fc0-5a2c-448c-95cf-1e86361c3610\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " Jul 03 20:46:51.622976 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.622934 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-bundle\") pod \"2b737fc0-5a2c-448c-95cf-1e86361c3610\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " Jul 03 20:46:51.622976 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.622959 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-util\") pod \"2b737fc0-5a2c-448c-95cf-1e86361c3610\" (UID: \"2b737fc0-5a2c-448c-95cf-1e86361c3610\") " Jul 03 20:46:51.623425 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.623397 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-bundle" (OuterVolumeSpecName: "bundle") pod "2b737fc0-5a2c-448c-95cf-1e86361c3610" (UID: "2b737fc0-5a2c-448c-95cf-1e86361c3610"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:51.624939 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.624915 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b737fc0-5a2c-448c-95cf-1e86361c3610-kube-api-access-sdkhr" (OuterVolumeSpecName: "kube-api-access-sdkhr") pod "2b737fc0-5a2c-448c-95cf-1e86361c3610" (UID: "2b737fc0-5a2c-448c-95cf-1e86361c3610"). InnerVolumeSpecName "kube-api-access-sdkhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:46:51.630806 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.630779 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-util" (OuterVolumeSpecName: "util") pod "2b737fc0-5a2c-448c-95cf-1e86361c3610" (UID: "2b737fc0-5a2c-448c-95cf-1e86361c3610"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:51.723569 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.723510 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:51.723569 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.723540 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b737fc0-5a2c-448c-95cf-1e86361c3610-util\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:51.723569 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:51.723551 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdkhr\" (UniqueName: \"kubernetes.io/projected/2b737fc0-5a2c-448c-95cf-1e86361c3610-kube-api-access-sdkhr\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:52.455458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.455430 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" Jul 03 20:46:52.455458 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.455452 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56" event={"ID":"2b737fc0-5a2c-448c-95cf-1e86361c3610","Type":"ContainerDied","Data":"6ba0151ed3d320c7f821624d862b31751ab4697c499809d010a0da7952190381"} Jul 03 20:46:52.455908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.455497 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ba0151ed3d320c7f821624d862b31751ab4697c499809d010a0da7952190381" Jul 03 20:46:52.585688 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.585665 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:52.629830 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.629792 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-bundle\") pod \"47890d2f-69ac-4d41-9dd7-1f755a28414e\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " Jul 03 20:46:52.629980 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.629865 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-util\") pod \"47890d2f-69ac-4d41-9dd7-1f755a28414e\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " Jul 03 20:46:52.629980 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.629904 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgkd2\" (UniqueName: \"kubernetes.io/projected/47890d2f-69ac-4d41-9dd7-1f755a28414e-kube-api-access-zgkd2\") pod \"47890d2f-69ac-4d41-9dd7-1f755a28414e\" (UID: \"47890d2f-69ac-4d41-9dd7-1f755a28414e\") " Jul 03 20:46:52.630435 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.630409 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-bundle" (OuterVolumeSpecName: "bundle") pod "47890d2f-69ac-4d41-9dd7-1f755a28414e" (UID: "47890d2f-69ac-4d41-9dd7-1f755a28414e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:52.631992 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.631969 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47890d2f-69ac-4d41-9dd7-1f755a28414e-kube-api-access-zgkd2" (OuterVolumeSpecName: "kube-api-access-zgkd2") pod "47890d2f-69ac-4d41-9dd7-1f755a28414e" (UID: "47890d2f-69ac-4d41-9dd7-1f755a28414e"). InnerVolumeSpecName "kube-api-access-zgkd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:46:52.635421 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.635385 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-util" (OuterVolumeSpecName: "util") pod "47890d2f-69ac-4d41-9dd7-1f755a28414e" (UID: "47890d2f-69ac-4d41-9dd7-1f755a28414e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:46:52.730495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.730428 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:52.730495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.730452 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/47890d2f-69ac-4d41-9dd7-1f755a28414e-util\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:52.730495 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:52.730461 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgkd2\" (UniqueName: \"kubernetes.io/projected/47890d2f-69ac-4d41-9dd7-1f755a28414e-kube-api-access-zgkd2\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:46:53.462531 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:53.462447 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" Jul 03 20:46:53.462531 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:53.462473 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm" event={"ID":"47890d2f-69ac-4d41-9dd7-1f755a28414e","Type":"ContainerDied","Data":"94028bacd91a6ef1d55257a1c3f63b3dc71388cbcf61ab4a86bc0820b5967e9f"} Jul 03 20:46:53.462531 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:46:53.462518 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94028bacd91a6ef1d55257a1c3f63b3dc71388cbcf61ab4a86bc0820b5967e9f" Jul 03 20:47:11.725276 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:11.725251 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:47:11.727746 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:11.725251 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:47:31.779815 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:31.779723 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54659b7845-rchdr"] Jul 03 20:47:34.663102 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663063 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t"] Jul 03 20:47:34.663573 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663549 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerName="util" Jul 03 20:47:34.663573 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663566 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerName="util" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663579 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerName="pull" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663590 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerName="pull" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663599 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerName="extract" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663609 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerName="extract" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663622 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerName="util" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663630 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerName="util" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663648 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerName="pull" Jul 03 20:47:34.663690 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663680 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerName="pull" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663693 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663702 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663712 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663720 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663734 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerName="pull" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663742 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerName="pull" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663755 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerName="util" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663782 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerName="util" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663792 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663801 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663814 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerName="pull" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663822 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerName="pull" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663840 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerName="util" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663848 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerName="util" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663928 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d7c9fbf-1a21-4f60-840f-ab563bda1e98" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663937 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="47890d2f-69ac-4d41-9dd7-1f755a28414e" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663949 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6c46d6-2abe-4693-ae94-608e27c0c2af" containerName="extract" Jul 03 20:47:34.664376 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.663959 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b737fc0-5a2c-448c-95cf-1e86361c3610" containerName="extract" Jul 03 20:47:34.667230 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.667211 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:34.670230 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.670210 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 20:47:34.670351 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.670234 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 20:47:34.670351 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.670296 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tc8gn\"" Jul 03 20:47:34.694023 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.693794 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t"] Jul 03 20:47:34.777604 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.777577 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e2dae57-3aa2-4cd8-b298-43c6564e1694-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-r995t\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:34.777731 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.777613 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzc67\" (UniqueName: \"kubernetes.io/projected/9e2dae57-3aa2-4cd8-b298-43c6564e1694-kube-api-access-tzc67\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-r995t\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:34.878737 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.878708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e2dae57-3aa2-4cd8-b298-43c6564e1694-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-r995t\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:34.878861 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.878742 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzc67\" (UniqueName: \"kubernetes.io/projected/9e2dae57-3aa2-4cd8-b298-43c6564e1694-kube-api-access-tzc67\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-r995t\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:34.879099 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.879081 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e2dae57-3aa2-4cd8-b298-43c6564e1694-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-r995t\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:34.890285 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.890264 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzc67\" (UniqueName: \"kubernetes.io/projected/9e2dae57-3aa2-4cd8-b298-43c6564e1694-kube-api-access-tzc67\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-r995t\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:34.979235 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:34.979149 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:35.113216 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:35.113192 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t"] Jul 03 20:47:35.115093 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:47:35.115054 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2dae57_3aa2_4cd8_b298_43c6564e1694.slice/crio-5925e085e2d0f066171129a28b77e236e191046fad2efe72ce81e6a58504e49c WatchSource:0}: Error finding container 5925e085e2d0f066171129a28b77e236e191046fad2efe72ce81e6a58504e49c: Status 404 returned error can't find the container with id 5925e085e2d0f066171129a28b77e236e191046fad2efe72ce81e6a58504e49c Jul 03 20:47:35.117378 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:35.117364 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:47:35.675122 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:35.675074 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" event={"ID":"9e2dae57-3aa2-4cd8-b298-43c6564e1694","Type":"ContainerStarted","Data":"5925e085e2d0f066171129a28b77e236e191046fad2efe72ce81e6a58504e49c"} Jul 03 20:47:40.708247 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:40.708213 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" event={"ID":"9e2dae57-3aa2-4cd8-b298-43c6564e1694","Type":"ContainerStarted","Data":"bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4"} Jul 03 20:47:40.708629 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:40.708365 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:40.727701 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:40.727653 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" podStartSLOduration=2.20364057 podStartE2EDuration="6.727638295s" totalImagesPullingTime="4.523997725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:47:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:47:40.725579744 +0000 UTC m=+632.223147415" watchObservedRunningTime="2026-07-03 20:47:40.727638295 +0000 UTC m=+632.225206015" Jul 03 20:47:51.716030 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:51.715997 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:47:56.800200 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:56.800142 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-54659b7845-rchdr" podUID="0cf2eedf-3d0a-4635-b41a-77b07184dd23" containerName="console" containerID="cri-o://0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d" gracePeriod=15 Jul 03 20:47:57.046912 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.046886 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:47:57.170583 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170518 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-oauth-serving-cert\") pod \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " Jul 03 20:47:57.170583 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170567 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xpfc\" (UniqueName: \"kubernetes.io/projected/0cf2eedf-3d0a-4635-b41a-77b07184dd23-kube-api-access-9xpfc\") pod \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " Jul 03 20:47:57.170745 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170587 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-oauth-config\") pod \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " Jul 03 20:47:57.170745 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170605 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-config\") pod \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " Jul 03 20:47:57.170745 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170638 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-trusted-ca-bundle\") pod \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " Jul 03 20:47:57.170745 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170680 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-serving-cert\") pod \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " Jul 03 20:47:57.170745 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170714 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-service-ca\") pod \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\" (UID: \"0cf2eedf-3d0a-4635-b41a-77b07184dd23\") " Jul 03 20:47:57.171049 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.170905 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0cf2eedf-3d0a-4635-b41a-77b07184dd23" (UID: "0cf2eedf-3d0a-4635-b41a-77b07184dd23"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:47:57.171113 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.171090 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0cf2eedf-3d0a-4635-b41a-77b07184dd23" (UID: "0cf2eedf-3d0a-4635-b41a-77b07184dd23"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:47:57.171226 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.171199 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-config" (OuterVolumeSpecName: "console-config") pod "0cf2eedf-3d0a-4635-b41a-77b07184dd23" (UID: "0cf2eedf-3d0a-4635-b41a-77b07184dd23"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:47:57.171291 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.171266 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-service-ca" (OuterVolumeSpecName: "service-ca") pod "0cf2eedf-3d0a-4635-b41a-77b07184dd23" (UID: "0cf2eedf-3d0a-4635-b41a-77b07184dd23"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:47:57.172985 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.172959 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0cf2eedf-3d0a-4635-b41a-77b07184dd23" (UID: "0cf2eedf-3d0a-4635-b41a-77b07184dd23"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:47:57.173085 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.172981 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0cf2eedf-3d0a-4635-b41a-77b07184dd23" (UID: "0cf2eedf-3d0a-4635-b41a-77b07184dd23"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:47:57.173085 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.172964 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf2eedf-3d0a-4635-b41a-77b07184dd23-kube-api-access-9xpfc" (OuterVolumeSpecName: "kube-api-access-9xpfc") pod "0cf2eedf-3d0a-4635-b41a-77b07184dd23" (UID: "0cf2eedf-3d0a-4635-b41a-77b07184dd23"). InnerVolumeSpecName "kube-api-access-9xpfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:47:57.271709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.271677 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:47:57.271709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.271703 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:47:57.271709 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.271712 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-oauth-serving-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:47:57.271966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.271721 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xpfc\" (UniqueName: \"kubernetes.io/projected/0cf2eedf-3d0a-4635-b41a-77b07184dd23-kube-api-access-9xpfc\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:47:57.271966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.271730 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-oauth-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:47:57.271966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.271739 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-console-config\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:47:57.271966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.271747 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cf2eedf-3d0a-4635-b41a-77b07184dd23-trusted-ca-bundle\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:47:57.790834 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.790802 2558 generic.go:356] "Generic (PLEG): container finished" podID="0cf2eedf-3d0a-4635-b41a-77b07184dd23" containerID="0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d" exitCode=0 Jul 03 20:47:57.791042 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.790854 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54659b7845-rchdr" Jul 03 20:47:57.791042 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.790892 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54659b7845-rchdr" event={"ID":"0cf2eedf-3d0a-4635-b41a-77b07184dd23","Type":"ContainerDied","Data":"0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d"} Jul 03 20:47:57.791042 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.790942 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54659b7845-rchdr" event={"ID":"0cf2eedf-3d0a-4635-b41a-77b07184dd23","Type":"ContainerDied","Data":"97dcbf766d8dfea0caf4ff5f662e6f67f14a02743a7f89052cea639682ddd339"} Jul 03 20:47:57.791042 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.790966 2558 scope.go:117] "RemoveContainer" containerID="0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d" Jul 03 20:47:57.800155 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.800136 2558 scope.go:117] "RemoveContainer" containerID="0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d" Jul 03 20:47:57.800407 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:47:57.800391 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d\": container with ID starting with 0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d not found: ID does not exist" containerID="0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d" Jul 03 20:47:57.800639 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.800415 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d"} err="failed to get container status \"0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d\": rpc error: code = NotFound desc = could not find container \"0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d\": container with ID starting with 0fce4c7f7a807de7e8cf7423f3cc890881748a3540f08746556a0360df4e423d not found: ID does not exist" Jul 03 20:47:57.812354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.812330 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54659b7845-rchdr"] Jul 03 20:47:57.816514 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:57.816491 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54659b7845-rchdr"] Jul 03 20:47:59.051015 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:47:59.050987 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf2eedf-3d0a-4635-b41a-77b07184dd23" path="/var/lib/kubelet/pods/0cf2eedf-3d0a-4635-b41a-77b07184dd23/volumes" Jul 03 20:48:04.246179 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.246139 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t"] Jul 03 20:48:04.246661 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.246373 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" podUID="9e2dae57-3aa2-4cd8-b298-43c6564e1694" containerName="manager" containerID="cri-o://bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4" gracePeriod=10 Jul 03 20:48:04.500519 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.500464 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:48:04.628803 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.628779 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e2dae57-3aa2-4cd8-b298-43c6564e1694-extensions-socket-volume\") pod \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " Jul 03 20:48:04.628926 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.628853 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzc67\" (UniqueName: \"kubernetes.io/projected/9e2dae57-3aa2-4cd8-b298-43c6564e1694-kube-api-access-tzc67\") pod \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\" (UID: \"9e2dae57-3aa2-4cd8-b298-43c6564e1694\") " Jul 03 20:48:04.629164 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.629142 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e2dae57-3aa2-4cd8-b298-43c6564e1694-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9e2dae57-3aa2-4cd8-b298-43c6564e1694" (UID: "9e2dae57-3aa2-4cd8-b298-43c6564e1694"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 20:48:04.630807 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.630785 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e2dae57-3aa2-4cd8-b298-43c6564e1694-kube-api-access-tzc67" (OuterVolumeSpecName: "kube-api-access-tzc67") pod "9e2dae57-3aa2-4cd8-b298-43c6564e1694" (UID: "9e2dae57-3aa2-4cd8-b298-43c6564e1694"). InnerVolumeSpecName "kube-api-access-tzc67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:48:04.730285 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.730247 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e2dae57-3aa2-4cd8-b298-43c6564e1694-extensions-socket-volume\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:48:04.730285 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.730282 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzc67\" (UniqueName: \"kubernetes.io/projected/9e2dae57-3aa2-4cd8-b298-43c6564e1694-kube-api-access-tzc67\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:48:04.827220 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.827188 2558 generic.go:356] "Generic (PLEG): container finished" podID="9e2dae57-3aa2-4cd8-b298-43c6564e1694" containerID="bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4" exitCode=0 Jul 03 20:48:04.827344 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.827229 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" Jul 03 20:48:04.827344 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.827271 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" event={"ID":"9e2dae57-3aa2-4cd8-b298-43c6564e1694","Type":"ContainerDied","Data":"bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4"} Jul 03 20:48:04.827344 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.827315 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t" event={"ID":"9e2dae57-3aa2-4cd8-b298-43c6564e1694","Type":"ContainerDied","Data":"5925e085e2d0f066171129a28b77e236e191046fad2efe72ce81e6a58504e49c"} Jul 03 20:48:04.827344 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.827338 2558 scope.go:117] "RemoveContainer" containerID="bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4" Jul 03 20:48:04.836874 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.836858 2558 scope.go:117] "RemoveContainer" containerID="bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4" Jul 03 20:48:04.837134 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:48:04.837115 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4\": container with ID starting with bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4 not found: ID does not exist" containerID="bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4" Jul 03 20:48:04.837190 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.837142 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4"} err="failed to get container status \"bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4\": rpc error: code = NotFound desc = could not find container \"bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4\": container with ID starting with bb8b73f0f17743eaa61b85fc6e6b9ad84d828477d56b5743a4b020404d44ecb4 not found: ID does not exist" Jul 03 20:48:04.848851 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.848832 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t"] Jul 03 20:48:04.857420 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:04.857400 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-r995t"] Jul 03 20:48:05.051275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:05.051244 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e2dae57-3aa2-4cd8-b298-43c6564e1694" path="/var/lib/kubelet/pods/9e2dae57-3aa2-4cd8-b298-43c6564e1694/volumes" Jul 03 20:48:12.961520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.961483 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ptk6v"] Jul 03 20:48:12.961908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.961869 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf2eedf-3d0a-4635-b41a-77b07184dd23" containerName="console" Jul 03 20:48:12.961908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.961882 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf2eedf-3d0a-4635-b41a-77b07184dd23" containerName="console" Jul 03 20:48:12.961908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.961900 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2dae57-3aa2-4cd8-b298-43c6564e1694" containerName="manager" Jul 03 20:48:12.961908 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.961907 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2dae57-3aa2-4cd8-b298-43c6564e1694" containerName="manager" Jul 03 20:48:12.962035 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.961981 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2dae57-3aa2-4cd8-b298-43c6564e1694" containerName="manager" Jul 03 20:48:12.962035 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.961990 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf2eedf-3d0a-4635-b41a-77b07184dd23" containerName="console" Jul 03 20:48:12.965624 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.965604 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:12.969107 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.969081 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 20:48:12.969966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.969943 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 03 20:48:12.969966 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.969951 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 20:48:12.970141 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.969967 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c9dnh\"" Jul 03 20:48:12.973073 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:12.973055 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ptk6v"] Jul 03 20:48:13.053899 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.053869 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ptk6v"] Jul 03 20:48:13.099778 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.099738 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/051ade03-ec58-4714-b447-e40aec0f1d19-config-file\") pod \"limitador-limitador-64c94fd64f-ptk6v\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:13.099924 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.099796 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjnhg\" (UniqueName: \"kubernetes.io/projected/051ade03-ec58-4714-b447-e40aec0f1d19-kube-api-access-wjnhg\") pod \"limitador-limitador-64c94fd64f-ptk6v\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:13.201135 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.201097 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/051ade03-ec58-4714-b447-e40aec0f1d19-config-file\") pod \"limitador-limitador-64c94fd64f-ptk6v\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:13.201306 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.201161 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjnhg\" (UniqueName: \"kubernetes.io/projected/051ade03-ec58-4714-b447-e40aec0f1d19-kube-api-access-wjnhg\") pod \"limitador-limitador-64c94fd64f-ptk6v\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:13.201867 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.201850 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/051ade03-ec58-4714-b447-e40aec0f1d19-config-file\") pod \"limitador-limitador-64c94fd64f-ptk6v\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:13.211299 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.211276 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjnhg\" (UniqueName: \"kubernetes.io/projected/051ade03-ec58-4714-b447-e40aec0f1d19-kube-api-access-wjnhg\") pod \"limitador-limitador-64c94fd64f-ptk6v\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:13.277187 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.277099 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:13.404072 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.404050 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ptk6v"] Jul 03 20:48:13.406266 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:48:13.406239 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod051ade03_ec58_4714_b447_e40aec0f1d19.slice/crio-3adaf8fc3b4146180faabdfa230ebb50c933c1c797ea6828aa8f4950a1dc2db1 WatchSource:0}: Error finding container 3adaf8fc3b4146180faabdfa230ebb50c933c1c797ea6828aa8f4950a1dc2db1: Status 404 returned error can't find the container with id 3adaf8fc3b4146180faabdfa230ebb50c933c1c797ea6828aa8f4950a1dc2db1 Jul 03 20:48:13.872112 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:13.872080 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" event={"ID":"051ade03-ec58-4714-b447-e40aec0f1d19","Type":"ContainerStarted","Data":"3adaf8fc3b4146180faabdfa230ebb50c933c1c797ea6828aa8f4950a1dc2db1"} Jul 03 20:48:16.889821 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:16.889790 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" event={"ID":"051ade03-ec58-4714-b447-e40aec0f1d19","Type":"ContainerStarted","Data":"20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c"} Jul 03 20:48:16.890252 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:16.889849 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:16.907566 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:16.907521 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" podStartSLOduration=2.244473324 podStartE2EDuration="4.907507071s" totalImagesPullingTime="2.663033747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:48:16.90555645 +0000 UTC m=+668.403124110" watchObservedRunningTime="2026-07-03 20:48:16.907507071 +0000 UTC m=+668.405074730" Jul 03 20:48:27.895154 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:27.895125 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:30.007165 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.007132 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ptk6v"] Jul 03 20:48:30.007585 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.007294 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" podUID="051ade03-ec58-4714-b447-e40aec0f1d19" containerName="limitador" containerID="cri-o://20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c" gracePeriod=30 Jul 03 20:48:30.554826 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.554804 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:30.742503 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.742407 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/051ade03-ec58-4714-b447-e40aec0f1d19-config-file\") pod \"051ade03-ec58-4714-b447-e40aec0f1d19\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " Jul 03 20:48:30.742503 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.742488 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjnhg\" (UniqueName: \"kubernetes.io/projected/051ade03-ec58-4714-b447-e40aec0f1d19-kube-api-access-wjnhg\") pod \"051ade03-ec58-4714-b447-e40aec0f1d19\" (UID: \"051ade03-ec58-4714-b447-e40aec0f1d19\") " Jul 03 20:48:30.742836 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.742803 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/051ade03-ec58-4714-b447-e40aec0f1d19-config-file" (OuterVolumeSpecName: "config-file") pod "051ade03-ec58-4714-b447-e40aec0f1d19" (UID: "051ade03-ec58-4714-b447-e40aec0f1d19"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 20:48:30.744534 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.744510 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/051ade03-ec58-4714-b447-e40aec0f1d19-kube-api-access-wjnhg" (OuterVolumeSpecName: "kube-api-access-wjnhg") pod "051ade03-ec58-4714-b447-e40aec0f1d19" (UID: "051ade03-ec58-4714-b447-e40aec0f1d19"). InnerVolumeSpecName "kube-api-access-wjnhg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:48:30.843797 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.843748 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjnhg\" (UniqueName: \"kubernetes.io/projected/051ade03-ec58-4714-b447-e40aec0f1d19-kube-api-access-wjnhg\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:48:30.843797 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.843791 2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/051ade03-ec58-4714-b447-e40aec0f1d19-config-file\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:48:30.961103 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.961070 2558 generic.go:356] "Generic (PLEG): container finished" podID="051ade03-ec58-4714-b447-e40aec0f1d19" containerID="20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c" exitCode=0 Jul 03 20:48:30.961222 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.961133 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" Jul 03 20:48:30.961222 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.961153 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" event={"ID":"051ade03-ec58-4714-b447-e40aec0f1d19","Type":"ContainerDied","Data":"20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c"} Jul 03 20:48:30.961292 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.961223 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ptk6v" event={"ID":"051ade03-ec58-4714-b447-e40aec0f1d19","Type":"ContainerDied","Data":"3adaf8fc3b4146180faabdfa230ebb50c933c1c797ea6828aa8f4950a1dc2db1"} Jul 03 20:48:30.961292 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.961245 2558 scope.go:117] "RemoveContainer" containerID="20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c" Jul 03 20:48:30.970167 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.970151 2558 scope.go:117] "RemoveContainer" containerID="20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c" Jul 03 20:48:30.970386 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:48:30.970371 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c\": container with ID starting with 20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c not found: ID does not exist" containerID="20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c" Jul 03 20:48:30.970432 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.970392 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c"} err="failed to get container status \"20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c\": rpc error: code = NotFound desc = could not find container \"20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c\": container with ID starting with 20958392826e4582cadf1dbd08de1b12d1afd23fca2abd98f039f3757f9a613c not found: ID does not exist" Jul 03 20:48:30.982010 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.981990 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ptk6v"] Jul 03 20:48:30.985870 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:30.985850 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ptk6v"] Jul 03 20:48:31.051649 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.051623 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="051ade03-ec58-4714-b447-e40aec0f1d19" path="/var/lib/kubelet/pods/051ade03-ec58-4714-b447-e40aec0f1d19/volumes" Jul 03 20:48:31.755414 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.755376 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-npkv8"] Jul 03 20:48:31.755786 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.755745 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="051ade03-ec58-4714-b447-e40aec0f1d19" containerName="limitador" Jul 03 20:48:31.755786 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.755774 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="051ade03-ec58-4714-b447-e40aec0f1d19" containerName="limitador" Jul 03 20:48:31.755943 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.755857 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="051ade03-ec58-4714-b447-e40aec0f1d19" containerName="limitador" Jul 03 20:48:31.760306 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.760290 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:31.762928 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.762908 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 03 20:48:31.763088 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.763071 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-w9qp8\"" Jul 03 20:48:31.767121 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.767103 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-npkv8"] Jul 03 20:48:31.953228 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.953199 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70990dd4-bc02-44f9-860e-67007986badb-data\") pod \"postgres-69fb4594cf-npkv8\" (UID: \"70990dd4-bc02-44f9-860e-67007986badb\") " pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:31.953381 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:31.953232 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4vxq\" (UniqueName: \"kubernetes.io/projected/70990dd4-bc02-44f9-860e-67007986badb-kube-api-access-g4vxq\") pod \"postgres-69fb4594cf-npkv8\" (UID: \"70990dd4-bc02-44f9-860e-67007986badb\") " pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:32.054370 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:32.054329 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4vxq\" (UniqueName: \"kubernetes.io/projected/70990dd4-bc02-44f9-860e-67007986badb-kube-api-access-g4vxq\") pod \"postgres-69fb4594cf-npkv8\" (UID: \"70990dd4-bc02-44f9-860e-67007986badb\") " pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:32.054838 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:32.054414 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70990dd4-bc02-44f9-860e-67007986badb-data\") pod \"postgres-69fb4594cf-npkv8\" (UID: \"70990dd4-bc02-44f9-860e-67007986badb\") " pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:32.054838 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:32.054697 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70990dd4-bc02-44f9-860e-67007986badb-data\") pod \"postgres-69fb4594cf-npkv8\" (UID: \"70990dd4-bc02-44f9-860e-67007986badb\") " pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:32.062757 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:32.062735 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4vxq\" (UniqueName: \"kubernetes.io/projected/70990dd4-bc02-44f9-860e-67007986badb-kube-api-access-g4vxq\") pod \"postgres-69fb4594cf-npkv8\" (UID: \"70990dd4-bc02-44f9-860e-67007986badb\") " pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:32.071425 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:32.071383 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:32.195061 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:32.195033 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-npkv8"] Jul 03 20:48:32.197075 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:48:32.197052 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70990dd4_bc02_44f9_860e_67007986badb.slice/crio-1b0d51d6e744f182e4339da30358d55afbe57582aff92a655c1eb4818bd18c33 WatchSource:0}: Error finding container 1b0d51d6e744f182e4339da30358d55afbe57582aff92a655c1eb4818bd18c33: Status 404 returned error can't find the container with id 1b0d51d6e744f182e4339da30358d55afbe57582aff92a655c1eb4818bd18c33 Jul 03 20:48:32.980244 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:32.980175 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-npkv8" event={"ID":"70990dd4-bc02-44f9-860e-67007986badb","Type":"ContainerStarted","Data":"1b0d51d6e744f182e4339da30358d55afbe57582aff92a655c1eb4818bd18c33"} Jul 03 20:48:38.008917 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:38.008876 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-npkv8" event={"ID":"70990dd4-bc02-44f9-860e-67007986badb","Type":"ContainerStarted","Data":"d1b9f67fad011fe9041dc151b9b2b1da672c79baa5e0fd41122f5c2786b8c4ad"} Jul 03 20:48:38.009378 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:38.008941 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:38.025746 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:38.025673 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-npkv8" podStartSLOduration=2.177994954 podStartE2EDuration="7.025659371s" totalImagesPullingTime="4.847664417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:48:38.024317659 +0000 UTC m=+689.521885318" watchObservedRunningTime="2026-07-03 20:48:38.025659371 +0000 UTC m=+689.523227029" Jul 03 20:48:44.041950 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:44.041923 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-npkv8" Jul 03 20:48:50.853321 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:50.853291 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9688748fd-phsh5"] Jul 03 20:48:51.021697 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.021665 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9688748fd-phsh5"] Jul 03 20:48:51.021875 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.021793 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.024331 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.024308 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 20:48:51.024477 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.024369 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 20:48:51.024477 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.024431 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cjs4z\"" Jul 03 20:48:51.040443 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.040421 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-t9mkz"] Jul 03 20:48:51.071816 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.071728 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.077922 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.077897 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-t9mkz"] Jul 03 20:48:51.098073 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.098049 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvrbb\" (UniqueName: \"kubernetes.io/projected/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-kube-api-access-fvrbb\") pod \"maas-controller-6c7c44bb7-t9mkz\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.098218 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.098093 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grrr\" (UniqueName: \"kubernetes.io/projected/070e2a73-571b-440d-8d47-8387d6767b01-kube-api-access-4grrr\") pod \"maas-controller-9688748fd-phsh5\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.098218 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.098133 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/070e2a73-571b-440d-8d47-8387d6767b01-webhook-cert\") pod \"maas-controller-9688748fd-phsh5\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.098359 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.098340 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-webhook-cert\") pod \"maas-controller-6c7c44bb7-t9mkz\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.199073 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.199000 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-webhook-cert\") pod \"maas-controller-6c7c44bb7-t9mkz\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.199073 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.199051 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvrbb\" (UniqueName: \"kubernetes.io/projected/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-kube-api-access-fvrbb\") pod \"maas-controller-6c7c44bb7-t9mkz\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.199236 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.199076 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4grrr\" (UniqueName: \"kubernetes.io/projected/070e2a73-571b-440d-8d47-8387d6767b01-kube-api-access-4grrr\") pod \"maas-controller-9688748fd-phsh5\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.199236 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.199094 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/070e2a73-571b-440d-8d47-8387d6767b01-webhook-cert\") pod \"maas-controller-9688748fd-phsh5\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.201488 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.201463 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/070e2a73-571b-440d-8d47-8387d6767b01-webhook-cert\") pod \"maas-controller-9688748fd-phsh5\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.201598 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.201488 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-webhook-cert\") pod \"maas-controller-6c7c44bb7-t9mkz\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.206900 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.206875 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grrr\" (UniqueName: \"kubernetes.io/projected/070e2a73-571b-440d-8d47-8387d6767b01-kube-api-access-4grrr\") pod \"maas-controller-9688748fd-phsh5\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.206984 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.206936 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvrbb\" (UniqueName: \"kubernetes.io/projected/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-kube-api-access-fvrbb\") pod \"maas-controller-6c7c44bb7-t9mkz\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.334108 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.334074 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:51.385325 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.385289 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:51.495469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.495436 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9688748fd-phsh5"] Jul 03 20:48:51.496009 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:48:51.495979 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod070e2a73_571b_440d_8d47_8387d6767b01.slice/crio-08dba680f0f281a11832fbf35e8bdfef97735d884f7bb5abb4f89fcf8073eb68 WatchSource:0}: Error finding container 08dba680f0f281a11832fbf35e8bdfef97735d884f7bb5abb4f89fcf8073eb68: Status 404 returned error can't find the container with id 08dba680f0f281a11832fbf35e8bdfef97735d884f7bb5abb4f89fcf8073eb68 Jul 03 20:48:51.553330 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:51.553306 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-t9mkz"] Jul 03 20:48:51.554718 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:48:51.554695 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2f6ffa_77dd_4f15_980c_933c8c49b5e5.slice/crio-b4698c89db7713423e168f7e45f8a4b9a1b9310598f31528da1e3caacfafd37f WatchSource:0}: Error finding container b4698c89db7713423e168f7e45f8a4b9a1b9310598f31528da1e3caacfafd37f: Status 404 returned error can't find the container with id b4698c89db7713423e168f7e45f8a4b9a1b9310598f31528da1e3caacfafd37f Jul 03 20:48:52.081177 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:52.081143 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9688748fd-phsh5" event={"ID":"070e2a73-571b-440d-8d47-8387d6767b01","Type":"ContainerStarted","Data":"08dba680f0f281a11832fbf35e8bdfef97735d884f7bb5abb4f89fcf8073eb68"} Jul 03 20:48:52.082439 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:52.082410 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" event={"ID":"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5","Type":"ContainerStarted","Data":"b4698c89db7713423e168f7e45f8a4b9a1b9310598f31528da1e3caacfafd37f"} Jul 03 20:48:54.093875 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:54.093840 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" event={"ID":"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5","Type":"ContainerStarted","Data":"911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d"} Jul 03 20:48:54.094314 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:54.093919 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:48:54.095267 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:54.095245 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9688748fd-phsh5" event={"ID":"070e2a73-571b-440d-8d47-8387d6767b01","Type":"ContainerStarted","Data":"e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0"} Jul 03 20:48:54.095377 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:54.095365 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:48:54.114562 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:54.114524 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" podStartSLOduration=1.119576328 podStartE2EDuration="3.114510663s" totalImagesPullingTime="1.994934335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:48:54.113268957 +0000 UTC m=+705.610836617" watchObservedRunningTime="2026-07-03 20:48:54.114510663 +0000 UTC m=+705.612078350" Jul 03 20:48:54.132923 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:48:54.132878 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9688748fd-phsh5" podStartSLOduration=2.09023729 podStartE2EDuration="4.132860351s" totalImagesPullingTime="2.042623061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:48:54.131129688 +0000 UTC m=+705.628697369" watchObservedRunningTime="2026-07-03 20:48:54.132860351 +0000 UTC m=+705.630428017" Jul 03 20:49:05.104876 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.104843 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:49:05.105242 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.105018 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:49:05.162378 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.162338 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9688748fd-phsh5"] Jul 03 20:49:05.162582 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.162555 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-9688748fd-phsh5" podUID="070e2a73-571b-440d-8d47-8387d6767b01" containerName="manager" containerID="cri-o://e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0" gracePeriod=30 Jul 03 20:49:05.400384 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.400360 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:49:05.520935 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.520903 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4grrr\" (UniqueName: \"kubernetes.io/projected/070e2a73-571b-440d-8d47-8387d6767b01-kube-api-access-4grrr\") pod \"070e2a73-571b-440d-8d47-8387d6767b01\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " Jul 03 20:49:05.521119 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.520961 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/070e2a73-571b-440d-8d47-8387d6767b01-webhook-cert\") pod \"070e2a73-571b-440d-8d47-8387d6767b01\" (UID: \"070e2a73-571b-440d-8d47-8387d6767b01\") " Jul 03 20:49:05.523044 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.523013 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/070e2a73-571b-440d-8d47-8387d6767b01-kube-api-access-4grrr" (OuterVolumeSpecName: "kube-api-access-4grrr") pod "070e2a73-571b-440d-8d47-8387d6767b01" (UID: "070e2a73-571b-440d-8d47-8387d6767b01"). InnerVolumeSpecName "kube-api-access-4grrr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:49:05.523138 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.523062 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/070e2a73-571b-440d-8d47-8387d6767b01-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "070e2a73-571b-440d-8d47-8387d6767b01" (UID: "070e2a73-571b-440d-8d47-8387d6767b01"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:49:05.621804 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.621777 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4grrr\" (UniqueName: \"kubernetes.io/projected/070e2a73-571b-440d-8d47-8387d6767b01-kube-api-access-4grrr\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:49:05.621804 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:05.621799 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/070e2a73-571b-440d-8d47-8387d6767b01-webhook-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:49:06.155965 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.155931 2558 generic.go:356] "Generic (PLEG): container finished" podID="070e2a73-571b-440d-8d47-8387d6767b01" containerID="e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0" exitCode=0 Jul 03 20:49:06.156354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.155971 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9688748fd-phsh5" Jul 03 20:49:06.156354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.156010 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9688748fd-phsh5" event={"ID":"070e2a73-571b-440d-8d47-8387d6767b01","Type":"ContainerDied","Data":"e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0"} Jul 03 20:49:06.156354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.156051 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9688748fd-phsh5" event={"ID":"070e2a73-571b-440d-8d47-8387d6767b01","Type":"ContainerDied","Data":"08dba680f0f281a11832fbf35e8bdfef97735d884f7bb5abb4f89fcf8073eb68"} Jul 03 20:49:06.156354 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.156071 2558 scope.go:117] "RemoveContainer" containerID="e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0" Jul 03 20:49:06.165304 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.165118 2558 scope.go:117] "RemoveContainer" containerID="e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0" Jul 03 20:49:06.165381 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:49:06.165364 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0\": container with ID starting with e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0 not found: ID does not exist" containerID="e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0" Jul 03 20:49:06.165419 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.165389 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0"} err="failed to get container status \"e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0\": rpc error: code = NotFound desc = could not find container \"e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0\": container with ID starting with e60ae3f7c7403716e452de505324bff619642e5f5d82994e12261c7a98990bc0 not found: ID does not exist" Jul 03 20:49:06.176947 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.176926 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9688748fd-phsh5"] Jul 03 20:49:06.180517 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:06.180496 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9688748fd-phsh5"] Jul 03 20:49:07.055381 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:07.055348 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="070e2a73-571b-440d-8d47-8387d6767b01" path="/var/lib/kubelet/pods/070e2a73-571b-440d-8d47-8387d6767b01/volumes" Jul 03 20:49:25.554983 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.554952 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-78bdbc9f8d-z54wh"] Jul 03 20:49:25.555598 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.555297 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="070e2a73-571b-440d-8d47-8387d6767b01" containerName="manager" Jul 03 20:49:25.555598 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.555310 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="070e2a73-571b-440d-8d47-8387d6767b01" containerName="manager" Jul 03 20:49:25.555598 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.555367 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="070e2a73-571b-440d-8d47-8387d6767b01" containerName="manager" Jul 03 20:49:25.560017 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.559998 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.563096 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.563075 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 03 20:49:25.563199 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.563099 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-h94dw\"" Jul 03 20:49:25.570686 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.570661 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-78bdbc9f8d-z54wh"] Jul 03 20:49:25.679911 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.679878 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d13720df-838d-46db-8c34-bfa952f4663b-maas-api-tls\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.679911 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.679908 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d13720df-838d-46db-8c34-bfa952f4663b-openshift-service-ca\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.680161 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.679938 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdwp8\" (UniqueName: \"kubernetes.io/projected/d13720df-838d-46db-8c34-bfa952f4663b-kube-api-access-zdwp8\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.781045 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.781009 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d13720df-838d-46db-8c34-bfa952f4663b-maas-api-tls\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.781045 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.781045 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d13720df-838d-46db-8c34-bfa952f4663b-openshift-service-ca\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.781244 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.781067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdwp8\" (UniqueName: \"kubernetes.io/projected/d13720df-838d-46db-8c34-bfa952f4663b-kube-api-access-zdwp8\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.781674 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.781647 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d13720df-838d-46db-8c34-bfa952f4663b-openshift-service-ca\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.783508 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.783473 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d13720df-838d-46db-8c34-bfa952f4663b-maas-api-tls\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.790111 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.790091 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdwp8\" (UniqueName: \"kubernetes.io/projected/d13720df-838d-46db-8c34-bfa952f4663b-kube-api-access-zdwp8\") pod \"maas-api-78bdbc9f8d-z54wh\" (UID: \"d13720df-838d-46db-8c34-bfa952f4663b\") " pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:25.873010 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:25.872951 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:26.007835 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.007800 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-78bdbc9f8d-z54wh"] Jul 03 20:49:26.010569 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:49:26.010536 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd13720df_838d_46db_8c34_bfa952f4663b.slice/crio-5cbb55baf977ddcfb22931571052a0c6264c51b471b5f32766e281d9870b6702 WatchSource:0}: Error finding container 5cbb55baf977ddcfb22931571052a0c6264c51b471b5f32766e281d9870b6702: Status 404 returned error can't find the container with id 5cbb55baf977ddcfb22931571052a0c6264c51b471b5f32766e281d9870b6702 Jul 03 20:49:26.101900 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.101874 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-x8pl2"] Jul 03 20:49:26.106357 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.106339 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.108879 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.108859 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 03 20:49:26.109222 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.109207 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vqqdh\"" Jul 03 20:49:26.117162 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.117143 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-x8pl2"] Jul 03 20:49:26.147132 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.147074 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nxcg8"] Jul 03 20:49:26.150790 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.150752 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.160089 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.160070 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nxcg8"] Jul 03 20:49:26.184840 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.184815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhxhz\" (UniqueName: \"kubernetes.io/projected/e4283f37-329c-4daf-9528-636b9bb18729-kube-api-access-bhxhz\") pod \"payload-pre-processing-7964457696-x8pl2\" (UID: \"e4283f37-329c-4daf-9528-636b9bb18729\") " pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.184956 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.184920 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4283f37-329c-4daf-9528-636b9bb18729-plugins-config-volume\") pod \"payload-pre-processing-7964457696-x8pl2\" (UID: \"e4283f37-329c-4daf-9528-636b9bb18729\") " pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.259657 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.259623 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-78bdbc9f8d-z54wh" event={"ID":"d13720df-838d-46db-8c34-bfa952f4663b","Type":"ContainerStarted","Data":"5cbb55baf977ddcfb22931571052a0c6264c51b471b5f32766e281d9870b6702"} Jul 03 20:49:26.286184 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.286006 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d86a36-e625-4b48-8261-e53315496ef5-plugins-config-volume\") pod \"payload-processing-58f86c7798-nxcg8\" (UID: \"85d86a36-e625-4b48-8261-e53315496ef5\") " pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.291744 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.286457 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t7kd\" (UniqueName: \"kubernetes.io/projected/85d86a36-e625-4b48-8261-e53315496ef5-kube-api-access-2t7kd\") pod \"payload-processing-58f86c7798-nxcg8\" (UID: \"85d86a36-e625-4b48-8261-e53315496ef5\") " pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.291744 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.286582 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4283f37-329c-4daf-9528-636b9bb18729-plugins-config-volume\") pod \"payload-pre-processing-7964457696-x8pl2\" (UID: \"e4283f37-329c-4daf-9528-636b9bb18729\") " pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.291744 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.286651 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhxhz\" (UniqueName: \"kubernetes.io/projected/e4283f37-329c-4daf-9528-636b9bb18729-kube-api-access-bhxhz\") pod \"payload-pre-processing-7964457696-x8pl2\" (UID: \"e4283f37-329c-4daf-9528-636b9bb18729\") " pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.291744 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.287904 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4283f37-329c-4daf-9528-636b9bb18729-plugins-config-volume\") pod \"payload-pre-processing-7964457696-x8pl2\" (UID: \"e4283f37-329c-4daf-9528-636b9bb18729\") " pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.294734 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.294708 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhxhz\" (UniqueName: \"kubernetes.io/projected/e4283f37-329c-4daf-9528-636b9bb18729-kube-api-access-bhxhz\") pod \"payload-pre-processing-7964457696-x8pl2\" (UID: \"e4283f37-329c-4daf-9528-636b9bb18729\") " pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.387632 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.387606 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t7kd\" (UniqueName: \"kubernetes.io/projected/85d86a36-e625-4b48-8261-e53315496ef5-kube-api-access-2t7kd\") pod \"payload-processing-58f86c7798-nxcg8\" (UID: \"85d86a36-e625-4b48-8261-e53315496ef5\") " pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.387792 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.387685 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d86a36-e625-4b48-8261-e53315496ef5-plugins-config-volume\") pod \"payload-processing-58f86c7798-nxcg8\" (UID: \"85d86a36-e625-4b48-8261-e53315496ef5\") " pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.388379 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.388356 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d86a36-e625-4b48-8261-e53315496ef5-plugins-config-volume\") pod \"payload-processing-58f86c7798-nxcg8\" (UID: \"85d86a36-e625-4b48-8261-e53315496ef5\") " pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.394725 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.394704 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t7kd\" (UniqueName: \"kubernetes.io/projected/85d86a36-e625-4b48-8261-e53315496ef5-kube-api-access-2t7kd\") pod \"payload-processing-58f86c7798-nxcg8\" (UID: \"85d86a36-e625-4b48-8261-e53315496ef5\") " pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.417684 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.417625 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:26.461614 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.461592 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:26.557250 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.557218 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-x8pl2"] Jul 03 20:49:26.558742 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:49:26.558709 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4283f37_329c_4daf_9528_636b9bb18729.slice/crio-e49ee7cbf92a03803aaeadfcce8d8883428467e269170cb84fb0fabc79f95103 WatchSource:0}: Error finding container e49ee7cbf92a03803aaeadfcce8d8883428467e269170cb84fb0fabc79f95103: Status 404 returned error can't find the container with id e49ee7cbf92a03803aaeadfcce8d8883428467e269170cb84fb0fabc79f95103 Jul 03 20:49:26.597469 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:26.597445 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-nxcg8"] Jul 03 20:49:26.599299 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:49:26.599276 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d86a36_e625_4b48_8261_e53315496ef5.slice/crio-d55f3c3c1a2af1a23b40bdfdf3645285745795f87402acb4002af1024bea0875 WatchSource:0}: Error finding container d55f3c3c1a2af1a23b40bdfdf3645285745795f87402acb4002af1024bea0875: Status 404 returned error can't find the container with id d55f3c3c1a2af1a23b40bdfdf3645285745795f87402acb4002af1024bea0875 Jul 03 20:49:27.273140 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:27.273084 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" event={"ID":"85d86a36-e625-4b48-8261-e53315496ef5","Type":"ContainerStarted","Data":"d55f3c3c1a2af1a23b40bdfdf3645285745795f87402acb4002af1024bea0875"} Jul 03 20:49:27.275979 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:27.275898 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" event={"ID":"e4283f37-329c-4daf-9528-636b9bb18729","Type":"ContainerStarted","Data":"e49ee7cbf92a03803aaeadfcce8d8883428467e269170cb84fb0fabc79f95103"} Jul 03 20:49:28.284352 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.284041 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-78bdbc9f8d-z54wh" event={"ID":"d13720df-838d-46db-8c34-bfa952f4663b","Type":"ContainerStarted","Data":"e347f91a12e9b31fbcceddff20cf5e414b3362c056cd8d41a401c0bde3f34f56"} Jul 03 20:49:28.284352 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.284197 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:28.285825 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.285800 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" event={"ID":"85d86a36-e625-4b48-8261-e53315496ef5","Type":"ContainerStarted","Data":"42ab7b97a66b59265d94e2ac7fe0e2e7ab04ef5fd63aeaf3c08acdcb3d41582e"} Jul 03 20:49:28.285951 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.285853 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:28.287275 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.287247 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" event={"ID":"e4283f37-329c-4daf-9528-636b9bb18729","Type":"ContainerStarted","Data":"f3b2c2672bd260539f4433677742541b5f66e503bba5975aa5c27298d1e54570"} Jul 03 20:49:28.287373 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.287292 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:49:28.301087 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.301053 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-78bdbc9f8d-z54wh" podStartSLOduration=1.835947081 podStartE2EDuration="3.301039519s" totalImagesPullingTime="1.465092438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:49:28.298849006 +0000 UTC m=+739.796416666" watchObservedRunningTime="2026-07-03 20:49:28.301039519 +0000 UTC m=+739.798607177" Jul 03 20:49:28.318752 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.318713 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" podStartSLOduration=0.9616803 podStartE2EDuration="2.318701842s" totalImagesPullingTime="1.357021542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:49:28.316750847 +0000 UTC m=+739.814318524" watchObservedRunningTime="2026-07-03 20:49:28.318701842 +0000 UTC m=+739.816269501" Jul 03 20:49:28.332915 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:28.332882 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" podStartSLOduration=1.018099248 podStartE2EDuration="2.332869944s" totalImagesPullingTime="1.314770696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:49:28.330444463 +0000 UTC m=+739.828012121" watchObservedRunningTime="2026-07-03 20:49:28.332869944 +0000 UTC m=+739.830437603" Jul 03 20:49:34.297797 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:34.297756 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-78bdbc9f8d-z54wh" Jul 03 20:49:39.294792 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:39.294750 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-nxcg8" Jul 03 20:49:39.295273 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:49:39.294882 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-x8pl2" Jul 03 20:50:15.654219 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.654189 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm"] Jul 03 20:50:15.657836 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.657819 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.660520 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.660499 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 03 20:50:15.661633 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.661614 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 03 20:50:15.661735 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.661647 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 03 20:50:15.661735 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.661669 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qjt4q\"" Jul 03 20:50:15.669154 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.669133 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm"] Jul 03 20:50:15.795870 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.795841 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-597m8\" (UniqueName: \"kubernetes.io/projected/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-kube-api-access-597m8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.796012 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.795886 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.796012 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.795962 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.796091 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.796010 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.796091 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.796039 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.796091 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.796084 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.896982 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.896955 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597m8\" (UniqueName: \"kubernetes.io/projected/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-kube-api-access-597m8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897137 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.896991 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897137 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.897016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897137 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.897038 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897137 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.897068 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897137 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.897096 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897415 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.897394 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897478 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.897421 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.897478 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.897460 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.899230 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.899211 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.899645 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.899627 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.904931 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.904879 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-597m8\" (UniqueName: \"kubernetes.io/projected/97fb9f2e-8a59-4cd1-a6d5-e9ca579128da-kube-api-access-597m8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jkmjm\" (UID: \"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:15.969072 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:15.969042 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:16.098296 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:16.098270 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm"] Jul 03 20:50:16.099877 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:50:16.099851 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97fb9f2e_8a59_4cd1_a6d5_e9ca579128da.slice/crio-a2982d536b043789ec106239f16aec54d84e5a38902debbc624ade975ea20b0a WatchSource:0}: Error finding container a2982d536b043789ec106239f16aec54d84e5a38902debbc624ade975ea20b0a: Status 404 returned error can't find the container with id a2982d536b043789ec106239f16aec54d84e5a38902debbc624ade975ea20b0a Jul 03 20:50:16.535631 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:16.535598 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" event={"ID":"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da","Type":"ContainerStarted","Data":"a2982d536b043789ec106239f16aec54d84e5a38902debbc624ade975ea20b0a"} Jul 03 20:50:17.541838 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:17.541804 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" event={"ID":"97fb9f2e-8a59-4cd1-a6d5-e9ca579128da","Type":"ContainerStarted","Data":"b289bb934933a9e66412182e634b80ab65f583a6180888ca2a52d8310453234c"} Jul 03 20:50:17.558720 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:17.558675 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" podStartSLOduration=1.588626557 podStartE2EDuration="2.558658683s" totalImagesPullingTime="970.032126ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:50:17.55771394 +0000 UTC m=+789.055281620" watchObservedRunningTime="2026-07-03 20:50:17.558658683 +0000 UTC m=+789.056226386" Jul 03 20:50:25.969304 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:25.969257 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:25.969712 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:25.969336 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:25.982179 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:25.982154 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:50:26.601873 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:50:26.601846 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jkmjm" Jul 03 20:52:11.760080 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:11.759978 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:52:11.766064 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:11.761552 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:52:36.882776 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:36.882731 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-t9mkz"] Jul 03 20:52:36.883348 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:36.882966 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" podUID="4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" containerName="manager" containerID="cri-o://911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d" gracePeriod=30 Jul 03 20:52:37.132223 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.132190 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:52:37.192301 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.192226 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-webhook-cert\") pod \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " Jul 03 20:52:37.192301 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.192274 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvrbb\" (UniqueName: \"kubernetes.io/projected/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-kube-api-access-fvrbb\") pod \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\" (UID: \"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5\") " Jul 03 20:52:37.194393 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.194362 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" (UID: "4c2f6ffa-77dd-4f15-980c-933c8c49b5e5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 20:52:37.194506 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.194445 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-kube-api-access-fvrbb" (OuterVolumeSpecName: "kube-api-access-fvrbb") pod "4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" (UID: "4c2f6ffa-77dd-4f15-980c-933c8c49b5e5"). InnerVolumeSpecName "kube-api-access-fvrbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 20:52:37.275311 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.275283 2558 generic.go:356] "Generic (PLEG): container finished" podID="4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" containerID="911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d" exitCode=0 Jul 03 20:52:37.275424 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.275327 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" Jul 03 20:52:37.275424 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.275342 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" event={"ID":"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5","Type":"ContainerDied","Data":"911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d"} Jul 03 20:52:37.275424 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.275395 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c7c44bb7-t9mkz" event={"ID":"4c2f6ffa-77dd-4f15-980c-933c8c49b5e5","Type":"ContainerDied","Data":"b4698c89db7713423e168f7e45f8a4b9a1b9310598f31528da1e3caacfafd37f"} Jul 03 20:52:37.275424 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.275424 2558 scope.go:117] "RemoveContainer" containerID="911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d" Jul 03 20:52:37.284536 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.284519 2558 scope.go:117] "RemoveContainer" containerID="911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d" Jul 03 20:52:37.284803 ip-10-0-131-199 kubenswrapper[2558]: E0703 20:52:37.284785 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d\": container with ID starting with 911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d not found: ID does not exist" containerID="911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d" Jul 03 20:52:37.284891 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.284813 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d"} err="failed to get container status \"911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d\": rpc error: code = NotFound desc = could not find container \"911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d\": container with ID starting with 911e97cf4d0e552f7cddd95b56a0608dbc2b056b3416199b51ca27dc59d8bc5d not found: ID does not exist" Jul 03 20:52:37.293576 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.293553 2558 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-webhook-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:52:37.293680 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.293580 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvrbb\" (UniqueName: \"kubernetes.io/projected/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5-kube-api-access-fvrbb\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 20:52:37.295814 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.295796 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-t9mkz"] Jul 03 20:52:37.299167 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:37.299148 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-t9mkz"] Jul 03 20:52:38.066176 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.066145 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-txjvx"] Jul 03 20:52:38.066528 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.066489 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" containerName="manager" Jul 03 20:52:38.066528 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.066500 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" containerName="manager" Jul 03 20:52:38.066599 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.066563 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" containerName="manager" Jul 03 20:52:38.070857 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.070841 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.073490 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.073469 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 03 20:52:38.074563 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.074546 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cjs4z\"" Jul 03 20:52:38.074646 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.074570 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 03 20:52:38.079523 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.079503 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-txjvx"] Jul 03 20:52:38.099608 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.099574 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tbsd\" (UniqueName: \"kubernetes.io/projected/eb784cff-9727-43cf-838a-af4b91f8a7bf-kube-api-access-6tbsd\") pod \"maas-controller-6c7c44bb7-txjvx\" (UID: \"eb784cff-9727-43cf-838a-af4b91f8a7bf\") " pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.099739 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.099617 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb784cff-9727-43cf-838a-af4b91f8a7bf-webhook-cert\") pod \"maas-controller-6c7c44bb7-txjvx\" (UID: \"eb784cff-9727-43cf-838a-af4b91f8a7bf\") " pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.200938 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.200906 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tbsd\" (UniqueName: \"kubernetes.io/projected/eb784cff-9727-43cf-838a-af4b91f8a7bf-kube-api-access-6tbsd\") pod \"maas-controller-6c7c44bb7-txjvx\" (UID: \"eb784cff-9727-43cf-838a-af4b91f8a7bf\") " pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.201092 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.200956 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb784cff-9727-43cf-838a-af4b91f8a7bf-webhook-cert\") pod \"maas-controller-6c7c44bb7-txjvx\" (UID: \"eb784cff-9727-43cf-838a-af4b91f8a7bf\") " pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.203384 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.203360 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb784cff-9727-43cf-838a-af4b91f8a7bf-webhook-cert\") pod \"maas-controller-6c7c44bb7-txjvx\" (UID: \"eb784cff-9727-43cf-838a-af4b91f8a7bf\") " pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.208864 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.208841 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tbsd\" (UniqueName: \"kubernetes.io/projected/eb784cff-9727-43cf-838a-af4b91f8a7bf-kube-api-access-6tbsd\") pod \"maas-controller-6c7c44bb7-txjvx\" (UID: \"eb784cff-9727-43cf-838a-af4b91f8a7bf\") " pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.382001 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.381914 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:38.510473 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.510449 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c7c44bb7-txjvx"] Jul 03 20:52:38.512100 ip-10-0-131-199 kubenswrapper[2558]: W0703 20:52:38.512075 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb784cff_9727_43cf_838a_af4b91f8a7bf.slice/crio-1c7fdafb28a9d11477d0fb641a1410c67d26ba43f3f88f4f7e3bcb8d16a540b2 WatchSource:0}: Error finding container 1c7fdafb28a9d11477d0fb641a1410c67d26ba43f3f88f4f7e3bcb8d16a540b2: Status 404 returned error can't find the container with id 1c7fdafb28a9d11477d0fb641a1410c67d26ba43f3f88f4f7e3bcb8d16a540b2 Jul 03 20:52:38.513318 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:38.513300 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 20:52:39.052976 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:39.052954 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2f6ffa-77dd-4f15-980c-933c8c49b5e5" path="/var/lib/kubelet/pods/4c2f6ffa-77dd-4f15-980c-933c8c49b5e5/volumes" Jul 03 20:52:39.288627 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:39.288591 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c7c44bb7-txjvx" event={"ID":"eb784cff-9727-43cf-838a-af4b91f8a7bf","Type":"ContainerStarted","Data":"ed02d03fb99c286ae90f046f6924ddf5f3b70f9893601380cd05ce3af6c08632"} Jul 03 20:52:39.289010 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:39.288650 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c7c44bb7-txjvx" event={"ID":"eb784cff-9727-43cf-838a-af4b91f8a7bf","Type":"ContainerStarted","Data":"1c7fdafb28a9d11477d0fb641a1410c67d26ba43f3f88f4f7e3bcb8d16a540b2"} Jul 03 20:52:39.289010 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:39.288730 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:52:39.304677 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:39.304603 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c7c44bb7-txjvx" podStartSLOduration=0.767430848 podStartE2EDuration="1.30458985s" totalImagesPullingTime="537.159002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 20:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 20:52:39.303236408 +0000 UTC m=+930.800804069" watchObservedRunningTime="2026-07-03 20:52:39.30458985 +0000 UTC m=+930.802157510" Jul 03 20:52:50.298860 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:52:50.298828 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c7c44bb7-txjvx" Jul 03 20:57:11.795628 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:57:11.795598 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 20:57:11.798812 ip-10-0-131-199 kubenswrapper[2558]: I0703 20:57:11.798786 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:00:00.144376 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.144343 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29718540-blnr9"] Jul 03 21:00:00.147715 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.147701 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" Jul 03 21:00:00.156804 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.156750 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718540-blnr9"] Jul 03 21:00:00.205687 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.205654 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwfms\" (UniqueName: \"kubernetes.io/projected/b7655090-869f-44e0-ac6e-30c52e90e928-kube-api-access-fwfms\") pod \"maas-api-key-cleanup-29718540-blnr9\" (UID: \"b7655090-869f-44e0-ac6e-30c52e90e928\") " pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" Jul 03 21:00:00.306937 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.306908 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwfms\" (UniqueName: \"kubernetes.io/projected/b7655090-869f-44e0-ac6e-30c52e90e928-kube-api-access-fwfms\") pod \"maas-api-key-cleanup-29718540-blnr9\" (UID: \"b7655090-869f-44e0-ac6e-30c52e90e928\") " pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" Jul 03 21:00:00.314942 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.314912 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwfms\" (UniqueName: \"kubernetes.io/projected/b7655090-869f-44e0-ac6e-30c52e90e928-kube-api-access-fwfms\") pod \"maas-api-key-cleanup-29718540-blnr9\" (UID: \"b7655090-869f-44e0-ac6e-30c52e90e928\") " pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" Jul 03 21:00:00.466885 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.466803 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" Jul 03 21:00:00.594775 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.594742 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718540-blnr9"] Jul 03 21:00:00.596348 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:00:00.596317 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7655090_869f_44e0_ac6e_30c52e90e928.slice/crio-b62635f2f8717c206bbf03e3a8389552cfafb7cbf198ffdceb0a6e83361cdd5f WatchSource:0}: Error finding container b62635f2f8717c206bbf03e3a8389552cfafb7cbf198ffdceb0a6e83361cdd5f: Status 404 returned error can't find the container with id b62635f2f8717c206bbf03e3a8389552cfafb7cbf198ffdceb0a6e83361cdd5f Jul 03 21:00:00.598109 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:00.598095 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 21:00:01.553806 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:01.553754 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" event={"ID":"b7655090-869f-44e0-ac6e-30c52e90e928","Type":"ContainerStarted","Data":"b62635f2f8717c206bbf03e3a8389552cfafb7cbf198ffdceb0a6e83361cdd5f"} Jul 03 21:00:04.573380 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:04.573349 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" event={"ID":"b7655090-869f-44e0-ac6e-30c52e90e928","Type":"ContainerStarted","Data":"147fcff6792a595123be1fa7f468f8ec3630904337e152fd8126cceadd86a13f"} Jul 03 21:00:04.597078 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:04.597025 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" podStartSLOduration=1.307079847 podStartE2EDuration="4.597008111s" totalImagesPullingTime="3.289928264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:00:04.593896988 +0000 UTC m=+1376.091464648" watchObservedRunningTime="2026-07-03 21:00:04.597008111 +0000 UTC m=+1376.094575770" Jul 03 21:00:05.580501 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:05.580471 2558 generic.go:356] "Generic (PLEG): container finished" podID="b7655090-869f-44e0-ac6e-30c52e90e928" containerID="147fcff6792a595123be1fa7f468f8ec3630904337e152fd8126cceadd86a13f" exitCode=0 Jul 03 21:00:05.581049 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:05.580542 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" event={"ID":"b7655090-869f-44e0-ac6e-30c52e90e928","Type":"ContainerDied","Data":"147fcff6792a595123be1fa7f468f8ec3630904337e152fd8126cceadd86a13f"} Jul 03 21:00:06.716837 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:06.716814 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" Jul 03 21:00:06.759615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:06.759594 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwfms\" (UniqueName: \"kubernetes.io/projected/b7655090-869f-44e0-ac6e-30c52e90e928-kube-api-access-fwfms\") pod \"b7655090-869f-44e0-ac6e-30c52e90e928\" (UID: \"b7655090-869f-44e0-ac6e-30c52e90e928\") " Jul 03 21:00:06.761501 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:06.761480 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7655090-869f-44e0-ac6e-30c52e90e928-kube-api-access-fwfms" (OuterVolumeSpecName: "kube-api-access-fwfms") pod "b7655090-869f-44e0-ac6e-30c52e90e928" (UID: "b7655090-869f-44e0-ac6e-30c52e90e928"). InnerVolumeSpecName "kube-api-access-fwfms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:00:06.860621 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:06.860553 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwfms\" (UniqueName: \"kubernetes.io/projected/b7655090-869f-44e0-ac6e-30c52e90e928-kube-api-access-fwfms\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:00:07.593787 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:07.593741 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" Jul 03 21:00:07.593787 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:07.593755 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718540-blnr9" event={"ID":"b7655090-869f-44e0-ac6e-30c52e90e928","Type":"ContainerDied","Data":"b62635f2f8717c206bbf03e3a8389552cfafb7cbf198ffdceb0a6e83361cdd5f"} Jul 03 21:00:07.593997 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:00:07.593825 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b62635f2f8717c206bbf03e3a8389552cfafb7cbf198ffdceb0a6e83361cdd5f" Jul 03 21:02:11.836667 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:02:11.836568 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:02:11.841474 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:02:11.841455 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:04:38.513723 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.513694 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv"] Jul 03 21:04:38.514151 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.514063 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7655090-869f-44e0-ac6e-30c52e90e928" containerName="cleanup" Jul 03 21:04:38.514151 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.514075 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7655090-869f-44e0-ac6e-30c52e90e928" containerName="cleanup" Jul 03 21:04:38.514151 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.514145 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7655090-869f-44e0-ac6e-30c52e90e928" containerName="cleanup" Jul 03 21:04:38.517301 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.517274 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.519606 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.519587 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-753c62a5-serving-cert\"" Jul 03 21:04:38.526858 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.526832 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv"] Jul 03 21:04:38.613124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.613062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18e14e9-b97f-4558-bac8-2ad49bf39127-openshift-service-ca\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.613124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.613102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-792pn\" (UniqueName: \"kubernetes.io/projected/c18e14e9-b97f-4558-bac8-2ad49bf39127-kube-api-access-792pn\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.613515 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.613172 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18e14e9-b97f-4558-bac8-2ad49bf39127-maas-api-tls\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.714038 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.714005 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18e14e9-b97f-4558-bac8-2ad49bf39127-openshift-service-ca\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.714038 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.714038 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-792pn\" (UniqueName: \"kubernetes.io/projected/c18e14e9-b97f-4558-bac8-2ad49bf39127-kube-api-access-792pn\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.714225 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.714058 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18e14e9-b97f-4558-bac8-2ad49bf39127-maas-api-tls\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.714679 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.714653 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18e14e9-b97f-4558-bac8-2ad49bf39127-openshift-service-ca\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.716635 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.716612 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18e14e9-b97f-4558-bac8-2ad49bf39127-maas-api-tls\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.722360 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.722338 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-792pn\" (UniqueName: \"kubernetes.io/projected/c18e14e9-b97f-4558-bac8-2ad49bf39127-kube-api-access-792pn\") pod \"maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.829611 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.829574 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:38.975561 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.975533 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv"] Jul 03 21:04:38.976530 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:04:38.976505 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18e14e9_b97f_4558_bac8_2ad49bf39127.slice/crio-a87132a06127e59393f920cd58376c43c3042e06c60046f871c0c44d4c0c606e WatchSource:0}: Error finding container a87132a06127e59393f920cd58376c43c3042e06c60046f871c0c44d4c0c606e: Status 404 returned error can't find the container with id a87132a06127e59393f920cd58376c43c3042e06c60046f871c0c44d4c0c606e Jul 03 21:04:38.999946 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:38.999914 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" event={"ID":"c18e14e9-b97f-4558-bac8-2ad49bf39127","Type":"ContainerStarted","Data":"a87132a06127e59393f920cd58376c43c3042e06c60046f871c0c44d4c0c606e"} Jul 03 21:04:40.005835 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:40.005801 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" event={"ID":"c18e14e9-b97f-4558-bac8-2ad49bf39127","Type":"ContainerStarted","Data":"3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22"} Jul 03 21:04:40.006217 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:40.005877 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:40.024179 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:40.024129 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" podStartSLOduration=1.575772081 podStartE2EDuration="2.024114462s" totalImagesPullingTime="448.342381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:04:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:04:40.022158714 +0000 UTC m=+1651.519726376" watchObservedRunningTime="2026-07-03 21:04:40.024114462 +0000 UTC m=+1651.521682163" Jul 03 21:04:43.688752 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.688710 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv"] Jul 03 21:04:43.689234 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.688979 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" podUID="c18e14e9-b97f-4558-bac8-2ad49bf39127" containerName="maas-api" containerID="cri-o://3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22" gracePeriod=30 Jul 03 21:04:43.936704 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.936682 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:43.950644 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.950550 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18e14e9-b97f-4558-bac8-2ad49bf39127-maas-api-tls\") pod \"c18e14e9-b97f-4558-bac8-2ad49bf39127\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " Jul 03 21:04:43.950644 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.950612 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18e14e9-b97f-4558-bac8-2ad49bf39127-openshift-service-ca\") pod \"c18e14e9-b97f-4558-bac8-2ad49bf39127\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " Jul 03 21:04:43.950644 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.950642 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-792pn\" (UniqueName: \"kubernetes.io/projected/c18e14e9-b97f-4558-bac8-2ad49bf39127-kube-api-access-792pn\") pod \"c18e14e9-b97f-4558-bac8-2ad49bf39127\" (UID: \"c18e14e9-b97f-4558-bac8-2ad49bf39127\") " Jul 03 21:04:43.951008 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.950981 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c18e14e9-b97f-4558-bac8-2ad49bf39127-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c18e14e9-b97f-4558-bac8-2ad49bf39127" (UID: "c18e14e9-b97f-4558-bac8-2ad49bf39127"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:04:43.953133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.953082 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c18e14e9-b97f-4558-bac8-2ad49bf39127-kube-api-access-792pn" (OuterVolumeSpecName: "kube-api-access-792pn") pod "c18e14e9-b97f-4558-bac8-2ad49bf39127" (UID: "c18e14e9-b97f-4558-bac8-2ad49bf39127"). InnerVolumeSpecName "kube-api-access-792pn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:04:43.953133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:43.953109 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c18e14e9-b97f-4558-bac8-2ad49bf39127-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c18e14e9-b97f-4558-bac8-2ad49bf39127" (UID: "c18e14e9-b97f-4558-bac8-2ad49bf39127"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:04:44.028293 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.028258 2558 generic.go:356] "Generic (PLEG): container finished" podID="c18e14e9-b97f-4558-bac8-2ad49bf39127" containerID="3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22" exitCode=0 Jul 03 21:04:44.028485 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.028327 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" Jul 03 21:04:44.028485 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.028349 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" event={"ID":"c18e14e9-b97f-4558-bac8-2ad49bf39127","Type":"ContainerDied","Data":"3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22"} Jul 03 21:04:44.028485 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.028416 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv" event={"ID":"c18e14e9-b97f-4558-bac8-2ad49bf39127","Type":"ContainerDied","Data":"a87132a06127e59393f920cd58376c43c3042e06c60046f871c0c44d4c0c606e"} Jul 03 21:04:44.028485 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.028446 2558 scope.go:117] "RemoveContainer" containerID="3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22" Jul 03 21:04:44.037889 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.037848 2558 scope.go:117] "RemoveContainer" containerID="3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22" Jul 03 21:04:44.038222 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:04:44.038184 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22\": container with ID starting with 3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22 not found: ID does not exist" containerID="3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22" Jul 03 21:04:44.038305 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.038234 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22"} err="failed to get container status \"3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22\": rpc error: code = NotFound desc = could not find container \"3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22\": container with ID starting with 3be71caabd9a464691d1db6e43659b4f2bbdf46f259b827921f782dae8f1ce22 not found: ID does not exist" Jul 03 21:04:44.051857 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.051833 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv"] Jul 03 21:04:44.052051 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.052027 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-792pn\" (UniqueName: \"kubernetes.io/projected/c18e14e9-b97f-4558-bac8-2ad49bf39127-kube-api-access-792pn\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:04:44.052051 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.052048 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c18e14e9-b97f-4558-bac8-2ad49bf39127-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:04:44.052238 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.052059 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c18e14e9-b97f-4558-bac8-2ad49bf39127-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:04:44.055381 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:44.055362 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-753c62a5-f4f475bb9-g9nmv"] Jul 03 21:04:45.051758 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:45.051722 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c18e14e9-b97f-4558-bac8-2ad49bf39127" path="/var/lib/kubelet/pods/c18e14e9-b97f-4558-bac8-2ad49bf39127/volumes" Jul 03 21:04:50.735703 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.735671 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5"] Jul 03 21:04:50.736244 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.736227 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c18e14e9-b97f-4558-bac8-2ad49bf39127" containerName="maas-api" Jul 03 21:04:50.736309 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.736246 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c18e14e9-b97f-4558-bac8-2ad49bf39127" containerName="maas-api" Jul 03 21:04:50.736371 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.736346 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c18e14e9-b97f-4558-bac8-2ad49bf39127" containerName="maas-api" Jul 03 21:04:50.741328 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.741308 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.744119 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.744069 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-ddaab74d-serving-cert\"" Jul 03 21:04:50.747950 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.747927 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5"] Jul 03 21:04:50.808230 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.808196 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fa16b8-dfc5-4bbd-8485-32dc57807a79-maas-api-tls\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.808458 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.808257 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgdhn\" (UniqueName: \"kubernetes.io/projected/31fa16b8-dfc5-4bbd-8485-32dc57807a79-kube-api-access-sgdhn\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.808458 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.808353 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fa16b8-dfc5-4bbd-8485-32dc57807a79-openshift-service-ca\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.909336 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.909287 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fa16b8-dfc5-4bbd-8485-32dc57807a79-openshift-service-ca\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.909522 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.909354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fa16b8-dfc5-4bbd-8485-32dc57807a79-maas-api-tls\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.909522 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.909388 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgdhn\" (UniqueName: \"kubernetes.io/projected/31fa16b8-dfc5-4bbd-8485-32dc57807a79-kube-api-access-sgdhn\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.910048 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.910025 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fa16b8-dfc5-4bbd-8485-32dc57807a79-openshift-service-ca\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.911798 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.911755 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fa16b8-dfc5-4bbd-8485-32dc57807a79-maas-api-tls\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:50.917723 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:50.917698 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgdhn\" (UniqueName: \"kubernetes.io/projected/31fa16b8-dfc5-4bbd-8485-32dc57807a79-kube-api-access-sgdhn\") pod \"maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:51.054408 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:51.054360 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:51.194798 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:51.194755 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5"] Jul 03 21:04:51.196408 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:04:51.196387 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fa16b8_dfc5_4bbd_8485_32dc57807a79.slice/crio-5966384f3db54371edc28acc7e57a01af9e226afd29167f513f9b092a2e14fb7 WatchSource:0}: Error finding container 5966384f3db54371edc28acc7e57a01af9e226afd29167f513f9b092a2e14fb7: Status 404 returned error can't find the container with id 5966384f3db54371edc28acc7e57a01af9e226afd29167f513f9b092a2e14fb7 Jul 03 21:04:52.073238 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:52.073201 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" event={"ID":"31fa16b8-dfc5-4bbd-8485-32dc57807a79","Type":"ContainerStarted","Data":"41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2"} Jul 03 21:04:52.073642 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:52.073261 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" event={"ID":"31fa16b8-dfc5-4bbd-8485-32dc57807a79","Type":"ContainerStarted","Data":"5966384f3db54371edc28acc7e57a01af9e226afd29167f513f9b092a2e14fb7"} Jul 03 21:04:52.073642 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:52.073289 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:52.090323 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:52.090273 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" podStartSLOduration=1.7419051300000001 podStartE2EDuration="2.090258014s" totalImagesPullingTime="348.352884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:04:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:04:52.089239554 +0000 UTC m=+1663.586807215" watchObservedRunningTime="2026-07-03 21:04:52.090258014 +0000 UTC m=+1663.587825692" Jul 03 21:04:56.024046 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.024012 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5"] Jul 03 21:04:56.024503 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.024236 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" podUID="31fa16b8-dfc5-4bbd-8485-32dc57807a79" containerName="maas-api" containerID="cri-o://41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2" gracePeriod=30 Jul 03 21:04:56.029628 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.029609 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:56.274391 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.274328 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:56.359322 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.359281 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fa16b8-dfc5-4bbd-8485-32dc57807a79-maas-api-tls\") pod \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " Jul 03 21:04:56.359502 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.359386 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgdhn\" (UniqueName: \"kubernetes.io/projected/31fa16b8-dfc5-4bbd-8485-32dc57807a79-kube-api-access-sgdhn\") pod \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " Jul 03 21:04:56.359573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.359551 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fa16b8-dfc5-4bbd-8485-32dc57807a79-openshift-service-ca\") pod \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\" (UID: \"31fa16b8-dfc5-4bbd-8485-32dc57807a79\") " Jul 03 21:04:56.359897 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.359875 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31fa16b8-dfc5-4bbd-8485-32dc57807a79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "31fa16b8-dfc5-4bbd-8485-32dc57807a79" (UID: "31fa16b8-dfc5-4bbd-8485-32dc57807a79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:04:56.361556 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.361531 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31fa16b8-dfc5-4bbd-8485-32dc57807a79-kube-api-access-sgdhn" (OuterVolumeSpecName: "kube-api-access-sgdhn") pod "31fa16b8-dfc5-4bbd-8485-32dc57807a79" (UID: "31fa16b8-dfc5-4bbd-8485-32dc57807a79"). InnerVolumeSpecName "kube-api-access-sgdhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:04:56.361556 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.361545 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31fa16b8-dfc5-4bbd-8485-32dc57807a79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "31fa16b8-dfc5-4bbd-8485-32dc57807a79" (UID: "31fa16b8-dfc5-4bbd-8485-32dc57807a79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:04:56.461207 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.461173 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/31fa16b8-dfc5-4bbd-8485-32dc57807a79-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:04:56.461207 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.461201 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgdhn\" (UniqueName: \"kubernetes.io/projected/31fa16b8-dfc5-4bbd-8485-32dc57807a79-kube-api-access-sgdhn\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:04:56.461207 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:56.461211 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/31fa16b8-dfc5-4bbd-8485-32dc57807a79-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:04:57.101898 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.101865 2558 generic.go:356] "Generic (PLEG): container finished" podID="31fa16b8-dfc5-4bbd-8485-32dc57807a79" containerID="41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2" exitCode=0 Jul 03 21:04:57.102366 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.101907 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" event={"ID":"31fa16b8-dfc5-4bbd-8485-32dc57807a79","Type":"ContainerDied","Data":"41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2"} Jul 03 21:04:57.102366 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.101954 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" event={"ID":"31fa16b8-dfc5-4bbd-8485-32dc57807a79","Type":"ContainerDied","Data":"5966384f3db54371edc28acc7e57a01af9e226afd29167f513f9b092a2e14fb7"} Jul 03 21:04:57.102366 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.101982 2558 scope.go:117] "RemoveContainer" containerID="41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2" Jul 03 21:04:57.102366 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.101909 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5" Jul 03 21:04:57.110934 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.110909 2558 scope.go:117] "RemoveContainer" containerID="41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2" Jul 03 21:04:57.111186 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:04:57.111166 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2\": container with ID starting with 41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2 not found: ID does not exist" containerID="41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2" Jul 03 21:04:57.111244 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.111193 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2"} err="failed to get container status \"41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2\": rpc error: code = NotFound desc = could not find container \"41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2\": container with ID starting with 41b73414843b946e2d7ac743774879d038b1ecf2e5a58c7a242604126dff0ce2 not found: ID does not exist" Jul 03 21:04:57.118647 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.118622 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5"] Jul 03 21:04:57.122052 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:57.122029 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-ddaab74d-5fb94559cd-964v5"] Jul 03 21:04:59.052714 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:04:59.052682 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31fa16b8-dfc5-4bbd-8485-32dc57807a79" path="/var/lib/kubelet/pods/31fa16b8-dfc5-4bbd-8485-32dc57807a79/volumes" Jul 03 21:05:07.967005 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:07.966969 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg"] Jul 03 21:05:07.967657 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:07.967624 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31fa16b8-dfc5-4bbd-8485-32dc57807a79" containerName="maas-api" Jul 03 21:05:07.967657 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:07.967644 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="31fa16b8-dfc5-4bbd-8485-32dc57807a79" containerName="maas-api" Jul 03 21:05:07.967803 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:07.967727 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="31fa16b8-dfc5-4bbd-8485-32dc57807a79" containerName="maas-api" Jul 03 21:05:07.973518 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:07.973487 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:07.976802 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:07.976691 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6478ce5b-openshift-default-dockercfg-vt7vm\"" Jul 03 21:05:07.987558 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:07.987524 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg"] Jul 03 21:05:08.058467 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058435 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-certs\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058467 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058473 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65c074e6-65c1-4600-8aa1-2ba05b774903-istiod-ca-cert\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058491 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-credential-socket\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058509 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-envoy\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058529 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkjpq\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-kube-api-access-wkjpq\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058582 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-token\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058686 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-socket\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058928 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058719 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-podinfo\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.058928 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.058746 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-data\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.159857 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.159819 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-podinfo\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160064 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.159883 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-data\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160064 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.159969 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-certs\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160064 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.159998 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65c074e6-65c1-4600-8aa1-2ba05b774903-istiod-ca-cert\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160064 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160028 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-credential-socket\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160064 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160056 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-envoy\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160089 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkjpq\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-kube-api-access-wkjpq\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160128 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-token\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160170 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-socket\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160511 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160417 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-data\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160511 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160430 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-certs\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160511 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160480 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-credential-socket\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160730 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160524 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-socket\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.160820 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.160792 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65c074e6-65c1-4600-8aa1-2ba05b774903-istiod-ca-cert\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.162403 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.162380 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-envoy\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.162507 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.162465 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-podinfo\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.168647 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.168623 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-token\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.169068 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.169048 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkjpq\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-kube-api-access-wkjpq\") pod \"e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.292788 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.292748 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:08.432203 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.432130 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg"] Jul 03 21:05:08.437157 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:05:08.437129 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c074e6_65c1_4600_8aa1_2ba05b774903.slice/crio-d93369c4c9f11dddb8c847827d641484ac6b7295b4f9a7493aa22a40834b9020 WatchSource:0}: Error finding container d93369c4c9f11dddb8c847827d641484ac6b7295b4f9a7493aa22a40834b9020: Status 404 returned error can't find the container with id d93369c4c9f11dddb8c847827d641484ac6b7295b4f9a7493aa22a40834b9020 Jul 03 21:05:08.439353 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:08.439333 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 21:05:09.170059 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.170024 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" event={"ID":"65c074e6-65c1-4600-8aa1-2ba05b774903","Type":"ContainerStarted","Data":"d93369c4c9f11dddb8c847827d641484ac6b7295b4f9a7493aa22a40834b9020"} Jul 03 21:05:09.424258 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.424175 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb"] Jul 03 21:05:09.428752 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.428719 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.432605 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.432335 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-6478ce5b-serving-cert\"" Jul 03 21:05:09.436200 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.436178 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb"] Jul 03 21:05:09.573097 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.573057 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd478931-ba44-4894-b85a-11d6de2eb03a-maas-api-tls\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.573297 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.573110 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd478931-ba44-4894-b85a-11d6de2eb03a-openshift-service-ca\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.573364 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.573287 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jszxs\" (UniqueName: \"kubernetes.io/projected/bd478931-ba44-4894-b85a-11d6de2eb03a-kube-api-access-jszxs\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.673911 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.673874 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd478931-ba44-4894-b85a-11d6de2eb03a-openshift-service-ca\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.674081 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.673975 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jszxs\" (UniqueName: \"kubernetes.io/projected/bd478931-ba44-4894-b85a-11d6de2eb03a-kube-api-access-jszxs\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.674081 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.674021 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd478931-ba44-4894-b85a-11d6de2eb03a-maas-api-tls\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.674546 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.674503 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd478931-ba44-4894-b85a-11d6de2eb03a-openshift-service-ca\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.676623 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.676599 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd478931-ba44-4894-b85a-11d6de2eb03a-maas-api-tls\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.686677 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.686653 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jszxs\" (UniqueName: \"kubernetes.io/projected/bd478931-ba44-4894-b85a-11d6de2eb03a-kube-api-access-jszxs\") pod \"maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.743509 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.743407 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:09.905759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:09.905730 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb"] Jul 03 21:05:09.907569 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:05:09.907541 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd478931_ba44_4894_b85a_11d6de2eb03a.slice/crio-814f829d0af17acea3024c0f2c4cf55592269706473cfa6fe68145ff9abb83d0 WatchSource:0}: Error finding container 814f829d0af17acea3024c0f2c4cf55592269706473cfa6fe68145ff9abb83d0: Status 404 returned error can't find the container with id 814f829d0af17acea3024c0f2c4cf55592269706473cfa6fe68145ff9abb83d0 Jul 03 21:05:10.178555 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:10.178517 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" event={"ID":"bd478931-ba44-4894-b85a-11d6de2eb03a","Type":"ContainerStarted","Data":"814f829d0af17acea3024c0f2c4cf55592269706473cfa6fe68145ff9abb83d0"} Jul 03 21:05:11.274260 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:11.274149 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:05:11.274260 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:11.274216 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:05:11.274260 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:11.274242 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:05:12.193067 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:12.193018 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" event={"ID":"bd478931-ba44-4894-b85a-11d6de2eb03a","Type":"ContainerStarted","Data":"6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e"} Jul 03 21:05:12.193275 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:12.193133 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:12.194537 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:12.194513 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" event={"ID":"65c074e6-65c1-4600-8aa1-2ba05b774903","Type":"ContainerStarted","Data":"deac84f22b0613b3e3478e5913f3a7761be10288b054fd66c945983b30d6c5dd"} Jul 03 21:05:12.211263 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:12.211212 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" podStartSLOduration=1.8471504520000002 podStartE2EDuration="3.211196284s" totalImagesPullingTime="1.364045832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:05:12.208126461 +0000 UTC m=+1683.705694143" watchObservedRunningTime="2026-07-03 21:05:12.211196284 +0000 UTC m=+1683.708763944" Jul 03 21:05:12.225564 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:12.225511 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" podStartSLOduration=2.391080178 podStartE2EDuration="5.22549708s" totalImagesPullingTime="2.834416902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:05:12.224841522 +0000 UTC m=+1683.722409205" watchObservedRunningTime="2026-07-03 21:05:12.22549708 +0000 UTC m=+1683.723064740" Jul 03 21:05:12.293727 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:12.293691 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:12.298618 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:12.298593 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:13.200262 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:13.200222 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:13.246191 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:13.246155 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:13.808201 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:13.808160 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg"] Jul 03 21:05:13.969641 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:13.969603 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb"] Jul 03 21:05:14.205698 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.205584 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" podUID="bd478931-ba44-4894-b85a-11d6de2eb03a" containerName="maas-api" containerID="cri-o://6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e" gracePeriod=30 Jul 03 21:05:14.205908 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.205884 2558 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" secret="" err="secret \"e2e-derive-6478ce5b-openshift-default-dockercfg-vt7vm\" not found" Jul 03 21:05:14.494409 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.494387 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:14.622318 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.622288 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd478931-ba44-4894-b85a-11d6de2eb03a-openshift-service-ca\") pod \"bd478931-ba44-4894-b85a-11d6de2eb03a\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " Jul 03 21:05:14.622487 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.622358 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd478931-ba44-4894-b85a-11d6de2eb03a-maas-api-tls\") pod \"bd478931-ba44-4894-b85a-11d6de2eb03a\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " Jul 03 21:05:14.622487 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.622376 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jszxs\" (UniqueName: \"kubernetes.io/projected/bd478931-ba44-4894-b85a-11d6de2eb03a-kube-api-access-jszxs\") pod \"bd478931-ba44-4894-b85a-11d6de2eb03a\" (UID: \"bd478931-ba44-4894-b85a-11d6de2eb03a\") " Jul 03 21:05:14.622634 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.622612 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd478931-ba44-4894-b85a-11d6de2eb03a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd478931-ba44-4894-b85a-11d6de2eb03a" (UID: "bd478931-ba44-4894-b85a-11d6de2eb03a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:05:14.624559 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.624528 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd478931-ba44-4894-b85a-11d6de2eb03a-kube-api-access-jszxs" (OuterVolumeSpecName: "kube-api-access-jszxs") pod "bd478931-ba44-4894-b85a-11d6de2eb03a" (UID: "bd478931-ba44-4894-b85a-11d6de2eb03a"). InnerVolumeSpecName "kube-api-access-jszxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:05:14.624682 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.624557 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd478931-ba44-4894-b85a-11d6de2eb03a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd478931-ba44-4894-b85a-11d6de2eb03a" (UID: "bd478931-ba44-4894-b85a-11d6de2eb03a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:05:14.723374 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.723329 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd478931-ba44-4894-b85a-11d6de2eb03a-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:14.723374 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.723367 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jszxs\" (UniqueName: \"kubernetes.io/projected/bd478931-ba44-4894-b85a-11d6de2eb03a-kube-api-access-jszxs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:14.723374 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:14.723379 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd478931-ba44-4894-b85a-11d6de2eb03a-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:15.212002 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.211965 2558 generic.go:356] "Generic (PLEG): container finished" podID="bd478931-ba44-4894-b85a-11d6de2eb03a" containerID="6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e" exitCode=0 Jul 03 21:05:15.212477 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.212013 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" Jul 03 21:05:15.212477 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.212054 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" event={"ID":"bd478931-ba44-4894-b85a-11d6de2eb03a","Type":"ContainerDied","Data":"6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e"} Jul 03 21:05:15.212477 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.212154 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb" event={"ID":"bd478931-ba44-4894-b85a-11d6de2eb03a","Type":"ContainerDied","Data":"814f829d0af17acea3024c0f2c4cf55592269706473cfa6fe68145ff9abb83d0"} Jul 03 21:05:15.212477 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.212175 2558 scope.go:117] "RemoveContainer" containerID="6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e" Jul 03 21:05:15.212477 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.212283 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" podUID="65c074e6-65c1-4600-8aa1-2ba05b774903" containerName="istio-proxy" containerID="cri-o://deac84f22b0613b3e3478e5913f3a7761be10288b054fd66c945983b30d6c5dd" gracePeriod=30 Jul 03 21:05:15.220918 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.220900 2558 scope.go:117] "RemoveContainer" containerID="6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e" Jul 03 21:05:15.221178 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:05:15.221158 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e\": container with ID starting with 6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e not found: ID does not exist" containerID="6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e" Jul 03 21:05:15.221223 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.221187 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e"} err="failed to get container status \"6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e\": rpc error: code = NotFound desc = could not find container \"6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e\": container with ID starting with 6c3a9abce2b3adbfcb47b2c6821e17600ec758ea24607604e38aee4230e8c66e not found: ID does not exist" Jul 03 21:05:15.251787 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.247003 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb"] Jul 03 21:05:15.261467 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:15.261427 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-6478ce5b-787dcbb5f4-6nfjb"] Jul 03 21:05:17.055224 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:17.055177 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd478931-ba44-4894-b85a-11d6de2eb03a" path="/var/lib/kubelet/pods/bd478931-ba44-4894-b85a-11d6de2eb03a/volumes" Jul 03 21:05:20.242452 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.242378 2558 generic.go:356] "Generic (PLEG): container finished" podID="65c074e6-65c1-4600-8aa1-2ba05b774903" containerID="deac84f22b0613b3e3478e5913f3a7761be10288b054fd66c945983b30d6c5dd" exitCode=0 Jul 03 21:05:20.242850 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.242455 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" event={"ID":"65c074e6-65c1-4600-8aa1-2ba05b774903","Type":"ContainerDied","Data":"deac84f22b0613b3e3478e5913f3a7761be10288b054fd66c945983b30d6c5dd"} Jul 03 21:05:20.277374 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.277349 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:20.377978 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.377897 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-socket\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.377978 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.377950 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-credential-socket\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.377978 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.377972 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65c074e6-65c1-4600-8aa1-2ba05b774903-istiod-ca-cert\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.378265 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378002 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-envoy\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.378265 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378024 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-data\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.378265 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378091 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-token\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.378265 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378127 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-podinfo\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.378265 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378178 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-certs\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.378265 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378206 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkjpq\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-kube-api-access-wkjpq\") pod \"65c074e6-65c1-4600-8aa1-2ba05b774903\" (UID: \"65c074e6-65c1-4600-8aa1-2ba05b774903\") " Jul 03 21:05:20.378265 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378250 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:05:20.378631 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378262 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:05:20.378631 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378465 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-credential-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.378631 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378484 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.378631 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378512 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:05:20.378631 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.378555 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c074e6-65c1-4600-8aa1-2ba05b774903-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:05:20.379174 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.379142 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-data" (OuterVolumeSpecName: "istio-data") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:05:20.380612 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.380583 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 21:05:20.380707 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.380610 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:05:20.380847 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.380829 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-kube-api-access-wkjpq" (OuterVolumeSpecName: "kube-api-access-wkjpq") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "kube-api-access-wkjpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:05:20.380897 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.380829 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-token" (OuterVolumeSpecName: "istio-token") pod "65c074e6-65c1-4600-8aa1-2ba05b774903" (UID: "65c074e6-65c1-4600-8aa1-2ba05b774903"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:05:20.479598 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.479570 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-podinfo\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.479598 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.479595 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-workload-certs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.479829 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.479604 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkjpq\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-kube-api-access-wkjpq\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.479829 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.479614 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65c074e6-65c1-4600-8aa1-2ba05b774903-istiod-ca-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.479829 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.479622 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-envoy\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.479829 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.479630 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-data\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:20.479829 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:20.479637 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65c074e6-65c1-4600-8aa1-2ba05b774903-istio-token\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:05:21.249075 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:21.249048 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" Jul 03 21:05:21.249500 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:21.249076 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg" event={"ID":"65c074e6-65c1-4600-8aa1-2ba05b774903","Type":"ContainerDied","Data":"d93369c4c9f11dddb8c847827d641484ac6b7295b4f9a7493aa22a40834b9020"} Jul 03 21:05:21.249500 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:21.249141 2558 scope.go:117] "RemoveContainer" containerID="deac84f22b0613b3e3478e5913f3a7761be10288b054fd66c945983b30d6c5dd" Jul 03 21:05:21.268024 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:21.267999 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg"] Jul 03 21:05:21.271818 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:21.271797 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6478ce5b-openshift-default-7cf88d748c-kn9wg"] Jul 03 21:05:23.052224 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:05:23.052189 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c074e6-65c1-4600-8aa1-2ba05b774903" path="/var/lib/kubelet/pods/65c074e6-65c1-4600-8aa1-2ba05b774903/volumes" Jul 03 21:06:54.336948 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.336911 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd"] Jul 03 21:06:54.337429 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.337415 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd478931-ba44-4894-b85a-11d6de2eb03a" containerName="maas-api" Jul 03 21:06:54.337472 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.337432 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd478931-ba44-4894-b85a-11d6de2eb03a" containerName="maas-api" Jul 03 21:06:54.337472 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.337457 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c074e6-65c1-4600-8aa1-2ba05b774903" containerName="istio-proxy" Jul 03 21:06:54.337472 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.337465 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c074e6-65c1-4600-8aa1-2ba05b774903" containerName="istio-proxy" Jul 03 21:06:54.337575 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.337564 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c074e6-65c1-4600-8aa1-2ba05b774903" containerName="istio-proxy" Jul 03 21:06:54.337610 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.337580 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd478931-ba44-4894-b85a-11d6de2eb03a" containerName="maas-api" Jul 03 21:06:54.340894 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.340875 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.343497 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.343478 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-54c4b733-openshift-default-dockercfg-7fsnr\"" Jul 03 21:06:54.360312 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.360287 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd"] Jul 03 21:06:54.442947 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.442919 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-certs\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443131 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.442966 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqd67\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-kube-api-access-mqd67\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443131 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.443020 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f705782-2470-411f-a12f-dac9c0b2d506-istiod-ca-cert\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443131 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.443062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-socket\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443131 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.443084 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-envoy\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443131 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.443122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f705782-2470-411f-a12f-dac9c0b2d506-istio-podinfo\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443422 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.443140 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-istio-token\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443422 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.443159 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-data\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.443422 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.443178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-credential-socket\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.543773 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.543735 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f705782-2470-411f-a12f-dac9c0b2d506-istio-podinfo\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.543910 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.543791 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-istio-token\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.543910 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.543826 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-data\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.543910 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.543856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-credential-socket\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.543910 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.543900 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-certs\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544093 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.543941 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqd67\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-kube-api-access-mqd67\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544093 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544027 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f705782-2470-411f-a12f-dac9c0b2d506-istiod-ca-cert\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544196 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544105 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-socket\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544196 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544134 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-envoy\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544306 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544262 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-data\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544400 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544379 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-certs\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544544 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544518 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-socket\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544676 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544633 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-credential-socket\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.544852 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.544837 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f705782-2470-411f-a12f-dac9c0b2d506-istiod-ca-cert\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.546193 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.546175 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f705782-2470-411f-a12f-dac9c0b2d506-istio-podinfo\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.546264 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.546214 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-envoy\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.551239 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.551220 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqd67\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-kube-api-access-mqd67\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.551341 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.551220 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-istio-token\") pod \"e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.650795 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.650690 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:54.793442 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.793419 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd"] Jul 03 21:06:54.794727 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:06:54.794695 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f705782_2470_411f_a12f_dac9c0b2d506.slice/crio-7cf63ab3502e65d3958f0c4294dda821d6ef989fedab299e6a6c32ed555b1a2f WatchSource:0}: Error finding container 7cf63ab3502e65d3958f0c4294dda821d6ef989fedab299e6a6c32ed555b1a2f: Status 404 returned error can't find the container with id 7cf63ab3502e65d3958f0c4294dda821d6ef989fedab299e6a6c32ed555b1a2f Jul 03 21:06:54.796948 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.796912 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:06:54.797044 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.796983 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:06:54.797044 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:54.797013 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:06:55.756057 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:55.756022 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" event={"ID":"8f705782-2470-411f-a12f-dac9c0b2d506","Type":"ContainerStarted","Data":"563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065"} Jul 03 21:06:55.756443 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:55.756076 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" event={"ID":"8f705782-2470-411f-a12f-dac9c0b2d506","Type":"ContainerStarted","Data":"7cf63ab3502e65d3958f0c4294dda821d6ef989fedab299e6a6c32ed555b1a2f"} Jul 03 21:06:55.776607 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:55.776557 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" podStartSLOduration=1.776540615 podStartE2EDuration="1.776540615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:06:55.774619084 +0000 UTC m=+1787.272186767" watchObservedRunningTime="2026-07-03 21:06:55.776540615 +0000 UTC m=+1787.274108274" Jul 03 21:06:56.651276 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:56.651239 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:56.656104 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:56.656081 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:56.761273 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:56.761245 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:56.767988 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:56.767961 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:06:57.827918 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:57.827885 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp"] Jul 03 21:06:57.831692 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:57.831672 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:57.834325 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:57.834308 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-54c4b733-serving-cert\"" Jul 03 21:06:57.841123 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:57.841096 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp"] Jul 03 21:06:57.974620 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:57.974588 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b6cd67f-9af8-411a-9386-26d76ebeff2f-maas-api-tls\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:57.974809 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:57.974693 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6cd67f-9af8-411a-9386-26d76ebeff2f-openshift-service-ca\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:57.974809 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:57.974731 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45pfw\" (UniqueName: \"kubernetes.io/projected/8b6cd67f-9af8-411a-9386-26d76ebeff2f-kube-api-access-45pfw\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.075163 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.075133 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b6cd67f-9af8-411a-9386-26d76ebeff2f-maas-api-tls\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.075319 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.075197 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6cd67f-9af8-411a-9386-26d76ebeff2f-openshift-service-ca\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.075319 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.075223 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45pfw\" (UniqueName: \"kubernetes.io/projected/8b6cd67f-9af8-411a-9386-26d76ebeff2f-kube-api-access-45pfw\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.075895 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.075870 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6cd67f-9af8-411a-9386-26d76ebeff2f-openshift-service-ca\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.077580 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.077554 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b6cd67f-9af8-411a-9386-26d76ebeff2f-maas-api-tls\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.087278 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.087226 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45pfw\" (UniqueName: \"kubernetes.io/projected/8b6cd67f-9af8-411a-9386-26d76ebeff2f-kube-api-access-45pfw\") pod \"maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.145223 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.145200 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:58.283355 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.283319 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp"] Jul 03 21:06:58.284725 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:06:58.284697 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6cd67f_9af8_411a_9386_26d76ebeff2f.slice/crio-f12d42f5c6a12be71f8c4ffb8b698210827c16d2f0c56e9bede621b334dfb752 WatchSource:0}: Error finding container f12d42f5c6a12be71f8c4ffb8b698210827c16d2f0c56e9bede621b334dfb752: Status 404 returned error can't find the container with id f12d42f5c6a12be71f8c4ffb8b698210827c16d2f0c56e9bede621b334dfb752 Jul 03 21:06:58.773817 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:58.773790 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" event={"ID":"8b6cd67f-9af8-411a-9386-26d76ebeff2f","Type":"ContainerStarted","Data":"f12d42f5c6a12be71f8c4ffb8b698210827c16d2f0c56e9bede621b334dfb752"} Jul 03 21:06:59.780619 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:59.780585 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" event={"ID":"8b6cd67f-9af8-411a-9386-26d76ebeff2f","Type":"ContainerStarted","Data":"588c09812d855ce4823709a4811c9745805ee58446dfbbe6f187a11322edfb7d"} Jul 03 21:06:59.781130 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:59.780788 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:06:59.796872 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:06:59.796832 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" podStartSLOduration=2.3502205800000002 podStartE2EDuration="2.796819325s" totalImagesPullingTime="446.598745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:06:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:06:59.796346979 +0000 UTC m=+1791.293914664" watchObservedRunningTime="2026-07-03 21:06:59.796819325 +0000 UTC m=+1791.294386984" Jul 03 21:07:05.790444 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:05.790364 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:07:09.671236 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.671204 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp"] Jul 03 21:07:09.671729 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.671436 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" podUID="8b6cd67f-9af8-411a-9386-26d76ebeff2f" containerName="maas-api" containerID="cri-o://588c09812d855ce4823709a4811c9745805ee58446dfbbe6f187a11322edfb7d" gracePeriod=30 Jul 03 21:07:09.837218 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.837186 2558 generic.go:356] "Generic (PLEG): container finished" podID="8b6cd67f-9af8-411a-9386-26d76ebeff2f" containerID="588c09812d855ce4823709a4811c9745805ee58446dfbbe6f187a11322edfb7d" exitCode=0 Jul 03 21:07:09.837429 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.837234 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" event={"ID":"8b6cd67f-9af8-411a-9386-26d76ebeff2f","Type":"ContainerDied","Data":"588c09812d855ce4823709a4811c9745805ee58446dfbbe6f187a11322edfb7d"} Jul 03 21:07:09.929804 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.929726 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:07:09.995616 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.995572 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45pfw\" (UniqueName: \"kubernetes.io/projected/8b6cd67f-9af8-411a-9386-26d76ebeff2f-kube-api-access-45pfw\") pod \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " Jul 03 21:07:09.995828 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.995681 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6cd67f-9af8-411a-9386-26d76ebeff2f-openshift-service-ca\") pod \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " Jul 03 21:07:09.995828 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.995722 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b6cd67f-9af8-411a-9386-26d76ebeff2f-maas-api-tls\") pod \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\" (UID: \"8b6cd67f-9af8-411a-9386-26d76ebeff2f\") " Jul 03 21:07:09.996086 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.996064 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b6cd67f-9af8-411a-9386-26d76ebeff2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b6cd67f-9af8-411a-9386-26d76ebeff2f" (UID: "8b6cd67f-9af8-411a-9386-26d76ebeff2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:07:09.997787 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.997732 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b6cd67f-9af8-411a-9386-26d76ebeff2f-kube-api-access-45pfw" (OuterVolumeSpecName: "kube-api-access-45pfw") pod "8b6cd67f-9af8-411a-9386-26d76ebeff2f" (UID: "8b6cd67f-9af8-411a-9386-26d76ebeff2f"). InnerVolumeSpecName "kube-api-access-45pfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:07:09.997902 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:09.997758 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b6cd67f-9af8-411a-9386-26d76ebeff2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b6cd67f-9af8-411a-9386-26d76ebeff2f" (UID: "8b6cd67f-9af8-411a-9386-26d76ebeff2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:07:10.096863 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.096809 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b6cd67f-9af8-411a-9386-26d76ebeff2f-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:10.096863 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.096842 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45pfw\" (UniqueName: \"kubernetes.io/projected/8b6cd67f-9af8-411a-9386-26d76ebeff2f-kube-api-access-45pfw\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:10.096863 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.096857 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b6cd67f-9af8-411a-9386-26d76ebeff2f-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:10.844097 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.844019 2558 generic.go:356] "Generic (PLEG): container finished" podID="8f705782-2470-411f-a12f-dac9c0b2d506" containerID="563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065" exitCode=0 Jul 03 21:07:10.844540 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.844094 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" event={"ID":"8f705782-2470-411f-a12f-dac9c0b2d506","Type":"ContainerDied","Data":"563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065"} Jul 03 21:07:10.844540 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.844493 2558 scope.go:117] "RemoveContainer" containerID="563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065" Jul 03 21:07:10.845527 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.845506 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" event={"ID":"8b6cd67f-9af8-411a-9386-26d76ebeff2f","Type":"ContainerDied","Data":"f12d42f5c6a12be71f8c4ffb8b698210827c16d2f0c56e9bede621b334dfb752"} Jul 03 21:07:10.845527 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.845520 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp" Jul 03 21:07:10.845668 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.845545 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:10.845668 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.845570 2558 scope.go:117] "RemoveContainer" containerID="588c09812d855ce4823709a4811c9745805ee58446dfbbe6f187a11322edfb7d" Jul 03 21:07:10.845668 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.845617 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:10.845668 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.845653 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:10.878936 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.878914 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp"] Jul 03 21:07:10.882267 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:10.882247 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-54c4b733-fbb65d899-zs2sp"] Jul 03 21:07:11.053240 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:11.053210 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b6cd67f-9af8-411a-9386-26d76ebeff2f" path="/var/lib/kubelet/pods/8b6cd67f-9af8-411a-9386-26d76ebeff2f/volumes" Jul 03 21:07:11.761305 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:11.761270 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:07:11.872064 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:11.872031 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" event={"ID":"8f705782-2470-411f-a12f-dac9c0b2d506","Type":"ContainerStarted","Data":"e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823"} Jul 03 21:07:11.898233 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:11.898208 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:07:11.901931 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:11.901908 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:07:12.652695 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:12.652651 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:07:12.652941 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:12.652887 2558 patch_prober.go:28] interesting pod/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 21:07:12.653027 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:12.652939 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 21:07:13.652043 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:13.652007 2558 patch_prober.go:28] interesting pod/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 21:07:13.652617 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:13.652587 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 21:07:14.651369 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:14.651333 2558 patch_prober.go:28] interesting pod/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 21:07:14.651713 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:14.651681 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 21:07:15.651444 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:15.651400 2558 patch_prober.go:28] interesting pod/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 03 21:07:15.651982 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:15.651467 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 03 21:07:15.902909 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:15.902822 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd"] Jul 03 21:07:15.903107 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:15.903007 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" containerID="cri-o://e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823" gracePeriod=30 Jul 03 21:07:21.159177 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.159147 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:07:21.297116 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297086 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-socket\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297146 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-envoy\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297168 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-credential-socket\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297196 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqd67\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-kube-api-access-mqd67\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297212 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f705782-2470-411f-a12f-dac9c0b2d506-istiod-ca-cert\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297230 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-istio-token\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297244 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-data\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297260 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f705782-2470-411f-a12f-dac9c0b2d506-istio-podinfo\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297308 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297302 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-certs\") pod \"8f705782-2470-411f-a12f-dac9c0b2d506\" (UID: \"8f705782-2470-411f-a12f-dac9c0b2d506\") " Jul 03 21:07:21.297710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297441 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:07:21.297710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297509 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:07:21.297710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297701 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:07:21.297926 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.297705 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f705782-2470-411f-a12f-dac9c0b2d506-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:07:21.298424 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.298390 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:07:21.299471 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.299441 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:07:21.299595 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.299460 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-kube-api-access-mqd67" (OuterVolumeSpecName: "kube-api-access-mqd67") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "kube-api-access-mqd67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:07:21.299595 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.299490 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:07:21.299595 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.299514 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f705782-2470-411f-a12f-dac9c0b2d506-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f705782-2470-411f-a12f-dac9c0b2d506" (UID: "8f705782-2470-411f-a12f-dac9c0b2d506"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 21:07:21.397976 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.397954 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.397976 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.397974 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-envoy\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.398124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.397983 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-credential-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.398124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.397992 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqd67\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-kube-api-access-mqd67\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.398124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.398000 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f705782-2470-411f-a12f-dac9c0b2d506-istiod-ca-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.398124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.398008 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f705782-2470-411f-a12f-dac9c0b2d506-istio-token\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.398124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.398015 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-istio-data\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.398124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.398022 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f705782-2470-411f-a12f-dac9c0b2d506-istio-podinfo\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.398124 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.398029 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f705782-2470-411f-a12f-dac9c0b2d506-workload-certs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:07:21.942464 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.942433 2558 generic.go:356] "Generic (PLEG): container finished" podID="8f705782-2470-411f-a12f-dac9c0b2d506" containerID="e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823" exitCode=0 Jul 03 21:07:21.942464 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.942468 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" event={"ID":"8f705782-2470-411f-a12f-dac9c0b2d506","Type":"ContainerDied","Data":"e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823"} Jul 03 21:07:21.942713 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.942490 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" Jul 03 21:07:21.942713 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.942529 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd" event={"ID":"8f705782-2470-411f-a12f-dac9c0b2d506","Type":"ContainerDied","Data":"7cf63ab3502e65d3958f0c4294dda821d6ef989fedab299e6a6c32ed555b1a2f"} Jul 03 21:07:21.942713 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.942552 2558 scope.go:117] "RemoveContainer" containerID="e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823" Jul 03 21:07:21.954359 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.954341 2558 scope.go:117] "RemoveContainer" containerID="563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065" Jul 03 21:07:21.962959 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.962942 2558 scope.go:117] "RemoveContainer" containerID="e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823" Jul 03 21:07:21.963226 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:07:21.963201 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823\": container with ID starting with e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823 not found: ID does not exist" containerID="e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823" Jul 03 21:07:21.963287 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.963239 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823"} err="failed to get container status \"e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823\": rpc error: code = NotFound desc = could not find container \"e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823\": container with ID starting with e7257183d9a9ec17063c697db6af02cecdaad3cf7395e8ef69336918098d2823 not found: ID does not exist" Jul 03 21:07:21.963287 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.963262 2558 scope.go:117] "RemoveContainer" containerID="563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065" Jul 03 21:07:21.963515 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:07:21.963492 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065\": container with ID starting with 563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065 not found: ID does not exist" containerID="563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065" Jul 03 21:07:21.963584 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.963532 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065"} err="failed to get container status \"563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065\": rpc error: code = NotFound desc = could not find container \"563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065\": container with ID starting with 563f77aa916a07331cb26ea2511ba863c3240cd60e8d32639eec548705a07065 not found: ID does not exist" Jul 03 21:07:21.969324 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.969302 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd"] Jul 03 21:07:21.973974 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:21.973955 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-54c4b733-openshift-default-6f78c7c959-q45bd"] Jul 03 21:07:23.051743 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:23.051706 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" path="/var/lib/kubelet/pods/8f705782-2470-411f-a12f-dac9c0b2d506/volumes" Jul 03 21:07:35.154208 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154155 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72"] Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154551 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154563 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154576 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154585 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154594 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b6cd67f-9af8-411a-9386-26d76ebeff2f" containerName="maas-api" Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154599 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b6cd67f-9af8-411a-9386-26d76ebeff2f" containerName="maas-api" Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154663 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" Jul 03 21:07:35.154726 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.154672 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b6cd67f-9af8-411a-9386-26d76ebeff2f" containerName="maas-api" Jul 03 21:07:35.157844 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.157826 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.160453 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.160426 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-73d964af-openshift-default-dockercfg-c7n25\"" Jul 03 21:07:35.165077 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.165052 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72"] Jul 03 21:07:35.317703 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.317669 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-socket\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.317703 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.317718 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-credential-socket\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.317982 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.317750 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knbpz\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-kube-api-access-knbpz\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.317982 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.317837 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-certs\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.317982 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.317899 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-data\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.317982 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.317931 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istiod-ca-cert\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.317982 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.317952 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-podinfo\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.318170 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.318015 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-envoy\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.318170 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.318038 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-token\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419379 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419296 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-data\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419379 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419337 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istiod-ca-cert\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419379 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419358 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-podinfo\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419665 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419390 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-envoy\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419665 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419410 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-token\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419665 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419438 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-socket\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419665 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419462 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-credential-socket\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419665 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knbpz\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-kube-api-access-knbpz\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.419665 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419546 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-certs\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.420009 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.419913 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-data\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.420054 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.420008 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-socket\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.420409 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.420029 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-credential-socket\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.420409 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.420375 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-certs\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.420600 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.420578 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istiod-ca-cert\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.427061 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.422674 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-envoy\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.427061 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.425610 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-podinfo\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.436005 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.435981 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knbpz\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-kube-api-access-knbpz\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.436901 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.436885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-token\") pod \"e2e-mt-73d964af-openshift-default-67d75f9687-h8d72\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.468855 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.468822 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:35.602224 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.602199 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72"] Jul 03 21:07:35.602913 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:07:35.602891 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7eb6a5_9956_4f63_b7bd_3a2f107f5418.slice/crio-28d34bda6b55fb329993b3115ab831cfadafce09653d5478293d11dd878afeea WatchSource:0}: Error finding container 28d34bda6b55fb329993b3115ab831cfadafce09653d5478293d11dd878afeea: Status 404 returned error can't find the container with id 28d34bda6b55fb329993b3115ab831cfadafce09653d5478293d11dd878afeea Jul 03 21:07:35.604986 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.604956 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:35.605058 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.605018 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:35.605058 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:35.605046 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:36.021027 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:36.020996 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" event={"ID":"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418","Type":"ContainerStarted","Data":"3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7"} Jul 03 21:07:36.021219 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:36.021050 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" event={"ID":"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418","Type":"ContainerStarted","Data":"28d34bda6b55fb329993b3115ab831cfadafce09653d5478293d11dd878afeea"} Jul 03 21:07:36.061616 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:36.061570 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" podStartSLOduration=1.061555358 podStartE2EDuration="1.061555358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:07:36.060702716 +0000 UTC m=+1827.558270388" watchObservedRunningTime="2026-07-03 21:07:36.061555358 +0000 UTC m=+1827.559123095" Jul 03 21:07:36.469753 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:36.469715 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:36.474788 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:36.474752 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:37.029936 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:37.029900 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:37.030994 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:37.030973 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:38.225840 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.225810 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz"] Jul 03 21:07:38.226545 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.226524 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f705782-2470-411f-a12f-dac9c0b2d506" containerName="istio-proxy" Jul 03 21:07:38.228724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.228694 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.231128 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.231102 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-73d964af-serving-cert\"" Jul 03 21:07:38.241557 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.241531 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz"] Jul 03 21:07:38.243116 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.243096 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.243226 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.243123 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61749940-c791-4677-876e-cdee2f3521ab-openshift-service-ca\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.243226 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.243175 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9s8n\" (UniqueName: \"kubernetes.io/projected/61749940-c791-4677-876e-cdee2f3521ab-kube-api-access-z9s8n\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.344156 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.344128 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9s8n\" (UniqueName: \"kubernetes.io/projected/61749940-c791-4677-876e-cdee2f3521ab-kube-api-access-z9s8n\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.344349 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.344229 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.344349 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.344258 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61749940-c791-4677-876e-cdee2f3521ab-openshift-service-ca\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.344479 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:07:38.344368 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-73d964af-serving-cert: secret "maas-api-e2e-mt-73d964af-serving-cert" not found Jul 03 21:07:38.344479 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:07:38.344434 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls podName:61749940-c791-4677-876e-cdee2f3521ab nodeName:}" failed. No retries permitted until 2026-07-03 21:07:38.844414023 +0000 UTC m=+1830.341981661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls") pod "maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" (UID: "61749940-c791-4677-876e-cdee2f3521ab") : secret "maas-api-e2e-mt-73d964af-serving-cert" not found Jul 03 21:07:38.344937 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.344915 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61749940-c791-4677-876e-cdee2f3521ab-openshift-service-ca\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.353310 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.353289 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9s8n\" (UniqueName: \"kubernetes.io/projected/61749940-c791-4677-876e-cdee2f3521ab-kube-api-access-z9s8n\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.849162 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.849117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:38.851593 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:38.851571 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls\") pod \"maas-api-e2e-mt-73d964af-5cbb568866-8l4nz\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:39.141668 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:39.141579 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:39.278893 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:39.278870 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz"] Jul 03 21:07:39.280147 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:07:39.280116 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61749940_c791_4677_876e_cdee2f3521ab.slice/crio-a6710f4547ab6e7409c76b9484d53a816fb8d691ebb5e4babce0c123f463506f WatchSource:0}: Error finding container a6710f4547ab6e7409c76b9484d53a816fb8d691ebb5e4babce0c123f463506f: Status 404 returned error can't find the container with id a6710f4547ab6e7409c76b9484d53a816fb8d691ebb5e4babce0c123f463506f Jul 03 21:07:40.046661 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:40.046626 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" event={"ID":"61749940-c791-4677-876e-cdee2f3521ab","Type":"ContainerStarted","Data":"b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150"} Jul 03 21:07:40.046850 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:40.046685 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" event={"ID":"61749940-c791-4677-876e-cdee2f3521ab","Type":"ContainerStarted","Data":"a6710f4547ab6e7409c76b9484d53a816fb8d691ebb5e4babce0c123f463506f"} Jul 03 21:07:40.046850 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:40.046725 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:40.069056 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:40.069004 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" podStartSLOduration=1.739036633 podStartE2EDuration="2.068990792s" totalImagesPullingTime="329.954159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:07:40.066357789 +0000 UTC m=+1831.563925473" watchObservedRunningTime="2026-07-03 21:07:40.068990792 +0000 UTC m=+1831.566558453" Jul 03 21:07:46.067513 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:46.067471 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:07:52.983101 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:52.983069 2558 patch_prober.go:28] interesting pod/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:33400->10.132.0.56:15021: read: connection reset by peer" start-of-body= Jul 03 21:07:52.983501 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:52.983177 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:33400->10.132.0.56:15021: read: connection reset by peer" Jul 03 21:07:53.116014 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:53.115984 2558 generic.go:356] "Generic (PLEG): container finished" podID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerID="3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7" exitCode=0 Jul 03 21:07:53.116190 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:53.116056 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" event={"ID":"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418","Type":"ContainerDied","Data":"3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7"} Jul 03 21:07:53.116432 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:53.116417 2558 scope.go:117] "RemoveContainer" containerID="3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7" Jul 03 21:07:53.117490 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:53.117464 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:53.117549 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:53.117535 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:53.117585 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:53.117569 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:07:54.124187 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:54.124152 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" event={"ID":"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418","Type":"ContainerStarted","Data":"fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f"} Jul 03 21:07:54.469845 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:54.469724 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:54.469996 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:54.469884 2558 patch_prober.go:28] interesting pod/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 21:07:54.469996 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:54.469939 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 21:07:55.470083 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:55.470045 2558 patch_prober.go:28] interesting pod/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 21:07:55.470468 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:55.470128 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 21:07:56.469551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:56.469512 2558 patch_prober.go:28] interesting pod/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 21:07:56.469897 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:56.469855 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 21:07:57.470215 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:57.470177 2558 patch_prober.go:28] interesting pod/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 03 21:07:57.470633 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:57.470231 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 03 21:07:58.472668 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:58.472639 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:58.473070 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:58.473020 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:07:58.473600 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:07:58.473580 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:08:05.263065 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.263031 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz"] Jul 03 21:08:05.263582 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.263284 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" podUID="61749940-c791-4677-876e-cdee2f3521ab" containerName="maas-api" containerID="cri-o://b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150" gracePeriod=30 Jul 03 21:08:05.503374 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.503349 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:08:05.595874 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.595834 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9s8n\" (UniqueName: \"kubernetes.io/projected/61749940-c791-4677-876e-cdee2f3521ab-kube-api-access-z9s8n\") pod \"61749940-c791-4677-876e-cdee2f3521ab\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " Jul 03 21:08:05.596056 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.595902 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls\") pod \"61749940-c791-4677-876e-cdee2f3521ab\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " Jul 03 21:08:05.596056 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.596007 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61749940-c791-4677-876e-cdee2f3521ab-openshift-service-ca\") pod \"61749940-c791-4677-876e-cdee2f3521ab\" (UID: \"61749940-c791-4677-876e-cdee2f3521ab\") " Jul 03 21:08:05.596329 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.596305 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61749940-c791-4677-876e-cdee2f3521ab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "61749940-c791-4677-876e-cdee2f3521ab" (UID: "61749940-c791-4677-876e-cdee2f3521ab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:08:05.597999 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.597965 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61749940-c791-4677-876e-cdee2f3521ab-kube-api-access-z9s8n" (OuterVolumeSpecName: "kube-api-access-z9s8n") pod "61749940-c791-4677-876e-cdee2f3521ab" (UID: "61749940-c791-4677-876e-cdee2f3521ab"). InnerVolumeSpecName "kube-api-access-z9s8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:08:05.597999 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.597972 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "61749940-c791-4677-876e-cdee2f3521ab" (UID: "61749940-c791-4677-876e-cdee2f3521ab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:08:05.696686 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.696651 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/61749940-c791-4677-876e-cdee2f3521ab-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:08:05.696686 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.696680 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9s8n\" (UniqueName: \"kubernetes.io/projected/61749940-c791-4677-876e-cdee2f3521ab-kube-api-access-z9s8n\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:08:05.696686 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:05.696690 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/61749940-c791-4677-876e-cdee2f3521ab-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:08:06.187872 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.187835 2558 generic.go:356] "Generic (PLEG): container finished" podID="61749940-c791-4677-876e-cdee2f3521ab" containerID="b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150" exitCode=0 Jul 03 21:08:06.188084 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.187887 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" Jul 03 21:08:06.188084 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.187924 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" event={"ID":"61749940-c791-4677-876e-cdee2f3521ab","Type":"ContainerDied","Data":"b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150"} Jul 03 21:08:06.188084 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.187989 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz" event={"ID":"61749940-c791-4677-876e-cdee2f3521ab","Type":"ContainerDied","Data":"a6710f4547ab6e7409c76b9484d53a816fb8d691ebb5e4babce0c123f463506f"} Jul 03 21:08:06.188084 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.188021 2558 scope.go:117] "RemoveContainer" containerID="b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150" Jul 03 21:08:06.197512 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.197495 2558 scope.go:117] "RemoveContainer" containerID="b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150" Jul 03 21:08:06.197816 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:08:06.197798 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150\": container with ID starting with b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150 not found: ID does not exist" containerID="b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150" Jul 03 21:08:06.197894 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.197830 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150"} err="failed to get container status \"b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150\": rpc error: code = NotFound desc = could not find container \"b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150\": container with ID starting with b7d129e5876abb774e73832a41f791c1d17f330b7156f78d084af793956d9150 not found: ID does not exist" Jul 03 21:08:06.209189 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.209163 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz"] Jul 03 21:08:06.212890 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:06.212867 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-73d964af-5cbb568866-8l4nz"] Jul 03 21:08:07.052319 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:07.052283 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61749940-c791-4677-876e-cdee2f3521ab" path="/var/lib/kubelet/pods/61749940-c791-4677-876e-cdee2f3521ab/volumes" Jul 03 21:08:58.676433 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:58.676398 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72"] Jul 03 21:08:58.677091 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:08:58.676608 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" containerID="cri-o://fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f" gracePeriod=30 Jul 03 21:09:03.937697 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.937674 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:09:03.998505 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998429 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-credential-socket\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998505 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998488 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knbpz\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-kube-api-access-knbpz\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998518 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-certs\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998533 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istiod-ca-cert\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998569 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-data\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998594 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-podinfo\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998633 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-socket\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998661 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-token\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.998724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998711 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-envoy\") pod \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\" (UID: \"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418\") " Jul 03 21:09:03.999115 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998719 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:09:03.999115 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998838 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:09:03.999115 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998935 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:09:03.999115 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.998989 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-credential-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:03.999115 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.999011 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-certs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:03.999115 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.999111 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:09:03.999439 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:03.999415 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:09:04.000890 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.000866 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:09:04.001016 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.000901 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-kube-api-access-knbpz" (OuterVolumeSpecName: "kube-api-access-knbpz") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "kube-api-access-knbpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:09:04.001016 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.000906 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 21:09:04.001016 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.000994 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" (UID: "8f7eb6a5-9956-4f63-b7bd-3a2f107f5418"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:09:04.100191 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.100161 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knbpz\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-kube-api-access-knbpz\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:04.100191 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.100184 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istiod-ca-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:04.100191 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.100193 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-data\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:04.100412 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.100201 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-podinfo\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:04.100412 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.100209 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-workload-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:04.100412 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.100219 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-token\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:04.100412 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.100227 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418-istio-envoy\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:09:04.492393 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.492359 2558 generic.go:356] "Generic (PLEG): container finished" podID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerID="fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f" exitCode=0 Jul 03 21:09:04.492393 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.492396 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" event={"ID":"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418","Type":"ContainerDied","Data":"fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f"} Jul 03 21:09:04.492650 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.492412 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" Jul 03 21:09:04.492650 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.492437 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72" event={"ID":"8f7eb6a5-9956-4f63-b7bd-3a2f107f5418","Type":"ContainerDied","Data":"28d34bda6b55fb329993b3115ab831cfadafce09653d5478293d11dd878afeea"} Jul 03 21:09:04.492650 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.492458 2558 scope.go:117] "RemoveContainer" containerID="fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f" Jul 03 21:09:04.504617 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.504570 2558 scope.go:117] "RemoveContainer" containerID="3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7" Jul 03 21:09:04.513573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.513557 2558 scope.go:117] "RemoveContainer" containerID="fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f" Jul 03 21:09:04.513818 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:09:04.513799 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f\": container with ID starting with fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f not found: ID does not exist" containerID="fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f" Jul 03 21:09:04.513906 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.513832 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f"} err="failed to get container status \"fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f\": rpc error: code = NotFound desc = could not find container \"fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f\": container with ID starting with fe6e20d64307be788016ddfbd593f7cde8b4c1a6d8ea572e4a6d908710bd605f not found: ID does not exist" Jul 03 21:09:04.513906 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.513857 2558 scope.go:117] "RemoveContainer" containerID="3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7" Jul 03 21:09:04.514130 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:09:04.514112 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7\": container with ID starting with 3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7 not found: ID does not exist" containerID="3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7" Jul 03 21:09:04.514168 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.514136 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7"} err="failed to get container status \"3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7\": rpc error: code = NotFound desc = could not find container \"3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7\": container with ID starting with 3f2137486ca5e1cc3b43fd6623e449576f3f42f6202ba8919189be32110f6bc7 not found: ID does not exist" Jul 03 21:09:04.515803 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.515751 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72"] Jul 03 21:09:04.521473 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:04.521453 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-73d964af-openshift-default-67d75f9687-h8d72"] Jul 03 21:09:05.051450 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:05.051422 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" path="/var/lib/kubelet/pods/8f7eb6a5-9956-4f63-b7bd-3a2f107f5418/volumes" Jul 03 21:09:52.973103 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973066 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z"] Jul 03 21:09:52.973581 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973482 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61749940-c791-4677-876e-cdee2f3521ab" containerName="maas-api" Jul 03 21:09:52.973581 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973494 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="61749940-c791-4677-876e-cdee2f3521ab" containerName="maas-api" Jul 03 21:09:52.973581 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973518 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" Jul 03 21:09:52.973581 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973523 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" Jul 03 21:09:52.973581 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973529 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" Jul 03 21:09:52.973581 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973536 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" Jul 03 21:09:52.973798 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973591 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="61749940-c791-4677-876e-cdee2f3521ab" containerName="maas-api" Jul 03 21:09:52.973798 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973602 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" Jul 03 21:09:52.973798 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.973609 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f7eb6a5-9956-4f63-b7bd-3a2f107f5418" containerName="istio-proxy" Jul 03 21:09:52.976800 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.976783 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:52.979410 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.979388 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-12615b-openshift-default-dockercfg-mvvl6\"" Jul 03 21:09:52.989389 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:52.989363 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z"] Jul 03 21:09:53.114342 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114304 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-socket\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114367 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-token\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114395 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-credential-socket\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114412 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-podinfo\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114434 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-certs\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114510 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmhcj\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-kube-api-access-fmhcj\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114546 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-envoy\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114831 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114569 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-data\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.114831 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.114587 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istiod-ca-cert\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215532 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-token\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215554 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-credential-socket\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215579 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-podinfo\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215608 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-certs\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215648 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmhcj\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-kube-api-access-fmhcj\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215683 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-envoy\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215712 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-data\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.215759 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215736 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istiod-ca-cert\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.216177 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.215856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-socket\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.216237 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.216181 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-certs\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.216237 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.216216 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-data\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.216349 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.216241 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-credential-socket\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.216408 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.216366 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-socket\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.216670 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.216646 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istiod-ca-cert\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.218133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.218100 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-envoy\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.218256 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.218222 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-podinfo\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.223967 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.223904 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-token\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.224077 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.224035 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmhcj\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-kube-api-access-fmhcj\") pod \"e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.288283 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.288253 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:53.429966 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.429932 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z"] Jul 03 21:09:53.431537 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:09:53.431514 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b6c0ab_d506_4fc4_8c9e_ad59aaf538f1.slice/crio-9a5d473f2b87986b56ed643c94da1042e336e6bace46bc63bd97a0f16c2b34df WatchSource:0}: Error finding container 9a5d473f2b87986b56ed643c94da1042e336e6bace46bc63bd97a0f16c2b34df: Status 404 returned error can't find the container with id 9a5d473f2b87986b56ed643c94da1042e336e6bace46bc63bd97a0f16c2b34df Jul 03 21:09:53.434247 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.434211 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:09:53.434363 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.434295 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:09:53.434363 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.434341 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:09:53.746817 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.746704 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" event={"ID":"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1","Type":"ContainerStarted","Data":"2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884"} Jul 03 21:09:53.746817 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.746787 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" event={"ID":"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1","Type":"ContainerStarted","Data":"9a5d473f2b87986b56ed643c94da1042e336e6bace46bc63bd97a0f16c2b34df"} Jul 03 21:09:53.766987 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:53.766941 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" podStartSLOduration=1.766928145 podStartE2EDuration="1.766928145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:09:53.765159249 +0000 UTC m=+1965.262726929" watchObservedRunningTime="2026-07-03 21:09:53.766928145 +0000 UTC m=+1965.264495805" Jul 03 21:09:54.288861 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:54.288827 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:54.294111 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:54.294085 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:54.752078 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:54.751979 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:54.753048 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:54.753027 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:09:55.842027 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:55.841992 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv"] Jul 03 21:09:55.846016 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:55.845994 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:55.848412 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:55.848388 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-12615b-serving-cert\"" Jul 03 21:09:55.853331 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:55.853306 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv"] Jul 03 21:09:55.938722 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:55.938689 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8rnv\" (UniqueName: \"kubernetes.io/projected/39514f03-0172-4b3a-8316-69b541bde4e2-kube-api-access-q8rnv\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:55.938932 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:55.938752 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39514f03-0172-4b3a-8316-69b541bde4e2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:55.938932 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:55.938900 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39514f03-0172-4b3a-8316-69b541bde4e2-maas-api-tls\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.039481 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.039449 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39514f03-0172-4b3a-8316-69b541bde4e2-maas-api-tls\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.039670 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.039490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8rnv\" (UniqueName: \"kubernetes.io/projected/39514f03-0172-4b3a-8316-69b541bde4e2-kube-api-access-q8rnv\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.039670 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.039560 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39514f03-0172-4b3a-8316-69b541bde4e2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.040295 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.040276 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39514f03-0172-4b3a-8316-69b541bde4e2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.042041 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.042018 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39514f03-0172-4b3a-8316-69b541bde4e2-maas-api-tls\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.050405 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.050379 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8rnv\" (UniqueName: \"kubernetes.io/projected/39514f03-0172-4b3a-8316-69b541bde4e2-kube-api-access-q8rnv\") pod \"maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.161467 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.161372 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:56.308357 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.308329 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv"] Jul 03 21:09:56.310345 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:09:56.310310 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39514f03_0172_4b3a_8316_69b541bde4e2.slice/crio-b29762ebf0e7f14c67003642b2e1d738fd2eb2d2e53778cd81b8a0b374636b4e WatchSource:0}: Error finding container b29762ebf0e7f14c67003642b2e1d738fd2eb2d2e53778cd81b8a0b374636b4e: Status 404 returned error can't find the container with id b29762ebf0e7f14c67003642b2e1d738fd2eb2d2e53778cd81b8a0b374636b4e Jul 03 21:09:56.765290 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:56.765260 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" event={"ID":"39514f03-0172-4b3a-8316-69b541bde4e2","Type":"ContainerStarted","Data":"b29762ebf0e7f14c67003642b2e1d738fd2eb2d2e53778cd81b8a0b374636b4e"} Jul 03 21:09:57.772642 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:57.772600 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" event={"ID":"39514f03-0172-4b3a-8316-69b541bde4e2","Type":"ContainerStarted","Data":"ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e"} Jul 03 21:09:57.773150 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:57.772660 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:09:57.790369 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:09:57.790318 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" podStartSLOduration=2.372100162 podStartE2EDuration="2.790302437s" totalImagesPullingTime="418.202275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:09:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:09:57.787615007 +0000 UTC m=+1969.285182668" watchObservedRunningTime="2026-07-03 21:09:57.790302437 +0000 UTC m=+1969.287870147" Jul 03 21:10:03.783558 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:03.783473 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:10:04.551425 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.551384 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx"] Jul 03 21:10:04.556214 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.556188 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.561063 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.561036 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-33b41b-serving-cert\"" Jul 03 21:10:04.592838 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.592694 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx"] Jul 03 21:10:04.615830 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.615800 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddb203fe-466a-440d-8ec1-2b75a3f33040-maas-api-tls\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.616002 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.615855 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl5rs\" (UniqueName: \"kubernetes.io/projected/ddb203fe-466a-440d-8ec1-2b75a3f33040-kube-api-access-bl5rs\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.616002 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.615916 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb203fe-466a-440d-8ec1-2b75a3f33040-openshift-service-ca\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.717050 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.717011 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddb203fe-466a-440d-8ec1-2b75a3f33040-maas-api-tls\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.717251 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.717073 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl5rs\" (UniqueName: \"kubernetes.io/projected/ddb203fe-466a-440d-8ec1-2b75a3f33040-kube-api-access-bl5rs\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.717251 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.717103 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb203fe-466a-440d-8ec1-2b75a3f33040-openshift-service-ca\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.717809 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.717753 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb203fe-466a-440d-8ec1-2b75a3f33040-openshift-service-ca\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.719407 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.719384 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddb203fe-466a-440d-8ec1-2b75a3f33040-maas-api-tls\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.726916 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.726891 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl5rs\" (UniqueName: \"kubernetes.io/projected/ddb203fe-466a-440d-8ec1-2b75a3f33040-kube-api-access-bl5rs\") pod \"maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:04.904820 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:04.904677 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:05.076652 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:05.075971 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx"] Jul 03 21:10:05.076652 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:10:05.076132 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb203fe_466a_440d_8ec1_2b75a3f33040.slice/crio-c8d3f8ea2242a335179ceb3e49c05adff80d21ad1fee7bc5e5c19a9ed4ff3f60 WatchSource:0}: Error finding container c8d3f8ea2242a335179ceb3e49c05adff80d21ad1fee7bc5e5c19a9ed4ff3f60: Status 404 returned error can't find the container with id c8d3f8ea2242a335179ceb3e49c05adff80d21ad1fee7bc5e5c19a9ed4ff3f60 Jul 03 21:10:05.815950 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:05.815918 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" event={"ID":"ddb203fe-466a-440d-8ec1-2b75a3f33040","Type":"ContainerStarted","Data":"39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a"} Jul 03 21:10:05.816122 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:05.815969 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" event={"ID":"ddb203fe-466a-440d-8ec1-2b75a3f33040","Type":"ContainerStarted","Data":"c8d3f8ea2242a335179ceb3e49c05adff80d21ad1fee7bc5e5c19a9ed4ff3f60"} Jul 03 21:10:05.816122 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:05.816047 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:05.832642 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:05.832594 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" podStartSLOduration=1.518958541 podStartE2EDuration="1.832579884s" totalImagesPullingTime="313.621343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:10:05.831527754 +0000 UTC m=+1977.329095415" watchObservedRunningTime="2026-07-03 21:10:05.832579884 +0000 UTC m=+1977.330147544" Jul 03 21:10:10.793954 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:10.793908 2558 patch_prober.go:28] interesting pod/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 03 21:10:10.794366 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:10.793973 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 03 21:10:11.826849 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:11.826819 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:10:11.855932 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:11.855900 2558 generic.go:356] "Generic (PLEG): container finished" podID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerID="2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884" exitCode=0 Jul 03 21:10:11.856105 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:11.855964 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" event={"ID":"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1","Type":"ContainerDied","Data":"2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884"} Jul 03 21:10:11.856395 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:11.856376 2558 scope.go:117] "RemoveContainer" containerID="2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884" Jul 03 21:10:11.857423 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:11.857395 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:10:11.857480 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:11.857472 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:10:11.857522 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:11.857505 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:10:12.867733 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:12.867688 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" event={"ID":"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1","Type":"ContainerStarted","Data":"c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8"} Jul 03 21:10:13.288972 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:13.288943 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:10:13.289246 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:13.289220 2558 patch_prober.go:28] interesting pod/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 21:10:13.289351 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:13.289284 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 21:10:14.289110 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:14.289076 2558 patch_prober.go:28] interesting pod/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 21:10:14.289507 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:14.289138 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 21:10:15.289494 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:15.289454 2558 patch_prober.go:28] interesting pod/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 21:10:15.289958 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:15.289514 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 21:10:16.288869 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:16.288836 2558 patch_prober.go:28] interesting pod/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 03 21:10:16.289068 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:16.288894 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 03 21:10:17.292591 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:17.292560 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:10:17.293108 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:17.292800 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:10:17.293495 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:10:17.293480 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:11:19.430679 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.430649 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv"] Jul 03 21:11:19.431237 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.430866 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" podUID="39514f03-0172-4b3a-8316-69b541bde4e2" containerName="maas-api" containerID="cri-o://ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e" gracePeriod=30 Jul 03 21:11:19.679648 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.679624 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:11:19.770144 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.770059 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39514f03-0172-4b3a-8316-69b541bde4e2-openshift-service-ca\") pod \"39514f03-0172-4b3a-8316-69b541bde4e2\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " Jul 03 21:11:19.770144 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.770112 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39514f03-0172-4b3a-8316-69b541bde4e2-maas-api-tls\") pod \"39514f03-0172-4b3a-8316-69b541bde4e2\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " Jul 03 21:11:19.770336 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.770172 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8rnv\" (UniqueName: \"kubernetes.io/projected/39514f03-0172-4b3a-8316-69b541bde4e2-kube-api-access-q8rnv\") pod \"39514f03-0172-4b3a-8316-69b541bde4e2\" (UID: \"39514f03-0172-4b3a-8316-69b541bde4e2\") " Jul 03 21:11:19.770476 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.770450 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39514f03-0172-4b3a-8316-69b541bde4e2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39514f03-0172-4b3a-8316-69b541bde4e2" (UID: "39514f03-0172-4b3a-8316-69b541bde4e2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:11:19.772313 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.772290 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39514f03-0172-4b3a-8316-69b541bde4e2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39514f03-0172-4b3a-8316-69b541bde4e2" (UID: "39514f03-0172-4b3a-8316-69b541bde4e2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:11:19.772454 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.772435 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39514f03-0172-4b3a-8316-69b541bde4e2-kube-api-access-q8rnv" (OuterVolumeSpecName: "kube-api-access-q8rnv") pod "39514f03-0172-4b3a-8316-69b541bde4e2" (UID: "39514f03-0172-4b3a-8316-69b541bde4e2"). InnerVolumeSpecName "kube-api-access-q8rnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:11:19.871389 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.871359 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8rnv\" (UniqueName: \"kubernetes.io/projected/39514f03-0172-4b3a-8316-69b541bde4e2-kube-api-access-q8rnv\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:19.871389 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.871382 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39514f03-0172-4b3a-8316-69b541bde4e2-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:19.871389 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:19.871393 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39514f03-0172-4b3a-8316-69b541bde4e2-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:20.244550 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.244514 2558 generic.go:356] "Generic (PLEG): container finished" podID="39514f03-0172-4b3a-8316-69b541bde4e2" containerID="ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e" exitCode=0 Jul 03 21:11:20.244745 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.244559 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" Jul 03 21:11:20.244745 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.244587 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" event={"ID":"39514f03-0172-4b3a-8316-69b541bde4e2","Type":"ContainerDied","Data":"ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e"} Jul 03 21:11:20.244745 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.244640 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv" event={"ID":"39514f03-0172-4b3a-8316-69b541bde4e2","Type":"ContainerDied","Data":"b29762ebf0e7f14c67003642b2e1d738fd2eb2d2e53778cd81b8a0b374636b4e"} Jul 03 21:11:20.244745 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.244668 2558 scope.go:117] "RemoveContainer" containerID="ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e" Jul 03 21:11:20.254203 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.254188 2558 scope.go:117] "RemoveContainer" containerID="ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e" Jul 03 21:11:20.254450 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:11:20.254432 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e\": container with ID starting with ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e not found: ID does not exist" containerID="ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e" Jul 03 21:11:20.254491 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.254459 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e"} err="failed to get container status \"ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e\": rpc error: code = NotFound desc = could not find container \"ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e\": container with ID starting with ade4ed83a3bc5a7c701913aa995f39cbd3121fd7b25539cc7ecb062f937e313e not found: ID does not exist" Jul 03 21:11:20.268098 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.268072 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv"] Jul 03 21:11:20.270318 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:20.270300 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-12615b-6c67675d7c-dkfmv"] Jul 03 21:11:21.052058 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:21.052028 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39514f03-0172-4b3a-8316-69b541bde4e2" path="/var/lib/kubelet/pods/39514f03-0172-4b3a-8316-69b541bde4e2/volumes" Jul 03 21:11:53.574290 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:53.574257 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z"] Jul 03 21:11:53.574903 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:53.574466 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" containerID="cri-o://c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8" gracePeriod=30 Jul 03 21:11:54.560638 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:54.560605 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx"] Jul 03 21:11:54.560867 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:54.560842 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" podUID="ddb203fe-466a-440d-8ec1-2b75a3f33040" containerName="maas-api" containerID="cri-o://39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a" gracePeriod=30 Jul 03 21:11:55.006998 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.006975 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:11:55.069746 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.069712 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl5rs\" (UniqueName: \"kubernetes.io/projected/ddb203fe-466a-440d-8ec1-2b75a3f33040-kube-api-access-bl5rs\") pod \"ddb203fe-466a-440d-8ec1-2b75a3f33040\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " Jul 03 21:11:55.069927 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.069774 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddb203fe-466a-440d-8ec1-2b75a3f33040-maas-api-tls\") pod \"ddb203fe-466a-440d-8ec1-2b75a3f33040\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " Jul 03 21:11:55.069927 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.069835 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb203fe-466a-440d-8ec1-2b75a3f33040-openshift-service-ca\") pod \"ddb203fe-466a-440d-8ec1-2b75a3f33040\" (UID: \"ddb203fe-466a-440d-8ec1-2b75a3f33040\") " Jul 03 21:11:55.070252 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.070225 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb203fe-466a-440d-8ec1-2b75a3f33040-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ddb203fe-466a-440d-8ec1-2b75a3f33040" (UID: "ddb203fe-466a-440d-8ec1-2b75a3f33040"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:11:55.071981 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.071955 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddb203fe-466a-440d-8ec1-2b75a3f33040-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ddb203fe-466a-440d-8ec1-2b75a3f33040" (UID: "ddb203fe-466a-440d-8ec1-2b75a3f33040"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:11:55.072116 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.072053 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb203fe-466a-440d-8ec1-2b75a3f33040-kube-api-access-bl5rs" (OuterVolumeSpecName: "kube-api-access-bl5rs") pod "ddb203fe-466a-440d-8ec1-2b75a3f33040" (UID: "ddb203fe-466a-440d-8ec1-2b75a3f33040"). InnerVolumeSpecName "kube-api-access-bl5rs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:11:55.171251 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.171175 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl5rs\" (UniqueName: \"kubernetes.io/projected/ddb203fe-466a-440d-8ec1-2b75a3f33040-kube-api-access-bl5rs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:55.171251 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.171202 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddb203fe-466a-440d-8ec1-2b75a3f33040-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:55.171251 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.171211 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddb203fe-466a-440d-8ec1-2b75a3f33040-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:55.431546 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.431466 2558 generic.go:356] "Generic (PLEG): container finished" podID="ddb203fe-466a-440d-8ec1-2b75a3f33040" containerID="39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a" exitCode=0 Jul 03 21:11:55.431546 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.431504 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" Jul 03 21:11:55.431749 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.431546 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" event={"ID":"ddb203fe-466a-440d-8ec1-2b75a3f33040","Type":"ContainerDied","Data":"39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a"} Jul 03 21:11:55.431749 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.431602 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx" event={"ID":"ddb203fe-466a-440d-8ec1-2b75a3f33040","Type":"ContainerDied","Data":"c8d3f8ea2242a335179ceb3e49c05adff80d21ad1fee7bc5e5c19a9ed4ff3f60"} Jul 03 21:11:55.431749 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.431625 2558 scope.go:117] "RemoveContainer" containerID="39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a" Jul 03 21:11:55.440880 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.440858 2558 scope.go:117] "RemoveContainer" containerID="39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a" Jul 03 21:11:55.441160 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:11:55.441141 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a\": container with ID starting with 39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a not found: ID does not exist" containerID="39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a" Jul 03 21:11:55.441229 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.441167 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a"} err="failed to get container status \"39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a\": rpc error: code = NotFound desc = could not find container \"39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a\": container with ID starting with 39e070bae26c4a4f97b31f0df83cfb3c42e99e7a389f3677f639cbfd9ba4841a not found: ID does not exist" Jul 03 21:11:55.452177 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.452156 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx"] Jul 03 21:11:55.455595 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:55.455575 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-33b41b-65c5b4c4b6-25ftx"] Jul 03 21:11:57.052021 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:57.051991 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddb203fe-466a-440d-8ec1-2b75a3f33040" path="/var/lib/kubelet/pods/ddb203fe-466a-440d-8ec1-2b75a3f33040/volumes" Jul 03 21:11:58.827528 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.827510 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:11:58.903836 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.903730 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istiod-ca-cert\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.903836 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.903818 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-certs\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904039 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.903855 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-data\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904039 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.903887 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-podinfo\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904039 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.903930 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-envoy\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904039 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.903973 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-socket\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904039 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904002 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-credential-socket\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904039 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904027 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-token\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904060 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmhcj\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-kube-api-access-fmhcj\") pod \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\" (UID: \"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1\") " Jul 03 21:11:58.904350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904217 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:11:58.904350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904234 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:11:58.904350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904275 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:11:58.904534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904401 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:11:58.904534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904412 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istiod-ca-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:58.904534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904433 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:58.904534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904447 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-credential-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:58.904889 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.904833 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:11:58.906151 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.906126 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:11:58.906243 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.906180 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:11:58.906243 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.906198 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-kube-api-access-fmhcj" (OuterVolumeSpecName: "kube-api-access-fmhcj") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "kube-api-access-fmhcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:11:58.906324 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:58.906310 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" (UID: "06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 21:11:59.005317 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.005286 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-workload-certs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:59.005317 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.005310 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-data\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:59.005317 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.005319 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-podinfo\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:59.005525 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.005327 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-envoy\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:59.005525 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.005336 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-istio-token\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:59.005525 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.005343 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmhcj\" (UniqueName: \"kubernetes.io/projected/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1-kube-api-access-fmhcj\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:11:59.456373 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.456338 2558 generic.go:356] "Generic (PLEG): container finished" podID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerID="c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8" exitCode=0 Jul 03 21:11:59.456549 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.456383 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" Jul 03 21:11:59.456549 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.456415 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" event={"ID":"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1","Type":"ContainerDied","Data":"c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8"} Jul 03 21:11:59.456549 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.456470 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z" event={"ID":"06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1","Type":"ContainerDied","Data":"9a5d473f2b87986b56ed643c94da1042e336e6bace46bc63bd97a0f16c2b34df"} Jul 03 21:11:59.456549 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.456493 2558 scope.go:117] "RemoveContainer" containerID="c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8" Jul 03 21:11:59.465609 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.465593 2558 scope.go:117] "RemoveContainer" containerID="2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884" Jul 03 21:11:59.473724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.473704 2558 scope.go:117] "RemoveContainer" containerID="c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8" Jul 03 21:11:59.474013 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:11:59.473994 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8\": container with ID starting with c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8 not found: ID does not exist" containerID="c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8" Jul 03 21:11:59.474114 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.474022 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8"} err="failed to get container status \"c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8\": rpc error: code = NotFound desc = could not find container \"c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8\": container with ID starting with c171f61ac5376a0f89498e98372f4579f5fb0075197869067cfec46ef643a8c8 not found: ID does not exist" Jul 03 21:11:59.474114 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.474039 2558 scope.go:117] "RemoveContainer" containerID="2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884" Jul 03 21:11:59.474339 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:11:59.474308 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884\": container with ID starting with 2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884 not found: ID does not exist" containerID="2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884" Jul 03 21:11:59.474380 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.474345 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884"} err="failed to get container status \"2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884\": rpc error: code = NotFound desc = could not find container \"2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884\": container with ID starting with 2a04987026559669bb25be399df4ecc59ab3edf430a4ba52f4fcd06faa2ef884 not found: ID does not exist" Jul 03 21:11:59.476451 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.476435 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z"] Jul 03 21:11:59.481528 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:11:59.481505 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-12615b-openshift-default-cf66c9c7f-m294z"] Jul 03 21:12:01.052926 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.052892 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" path="/var/lib/kubelet/pods/06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1/volumes" Jul 03 21:12:01.071980 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.071949 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6"] Jul 03 21:12:01.072510 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072494 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" Jul 03 21:12:01.072573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072513 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" Jul 03 21:12:01.072573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072527 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39514f03-0172-4b3a-8316-69b541bde4e2" containerName="maas-api" Jul 03 21:12:01.072573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072535 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="39514f03-0172-4b3a-8316-69b541bde4e2" containerName="maas-api" Jul 03 21:12:01.072573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072550 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb203fe-466a-440d-8ec1-2b75a3f33040" containerName="maas-api" Jul 03 21:12:01.072573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072559 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb203fe-466a-440d-8ec1-2b75a3f33040" containerName="maas-api" Jul 03 21:12:01.072573 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072571 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" Jul 03 21:12:01.072887 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072580 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" Jul 03 21:12:01.072887 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072665 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" Jul 03 21:12:01.072887 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072677 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="39514f03-0172-4b3a-8316-69b541bde4e2" containerName="maas-api" Jul 03 21:12:01.072887 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.072686 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb203fe-466a-440d-8ec1-2b75a3f33040" containerName="maas-api" Jul 03 21:12:01.077595 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.077572 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.080075 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.080055 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a176cd-openshift-default-dockercfg-szxtj\"" Jul 03 21:12:01.086698 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.086674 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6"] Jul 03 21:12:01.123713 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.123679 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-credential-socket\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.123886 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.123728 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-data\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.123886 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.123756 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz8fh\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-kube-api-access-hz8fh\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.123886 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.123863 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-token\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.124011 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.123914 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istiod-ca-cert\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.124011 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.123954 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-podinfo\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.124011 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.123988 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-socket\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.124116 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.124030 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-envoy\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.124116 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.124062 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-certs\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.224949 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.224914 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-data\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225119 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.224958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz8fh\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-kube-api-access-hz8fh\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225119 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225005 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-token\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225119 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istiod-ca-cert\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225119 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225077 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-podinfo\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225119 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225109 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-socket\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225424 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225154 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-envoy\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225424 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225185 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-certs\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225424 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225213 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-credential-socket\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225424 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225326 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-data\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225649 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225534 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-credential-socket\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225649 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225594 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-socket\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225649 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225635 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-certs\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.225830 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.225801 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istiod-ca-cert\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.227372 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.227351 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-envoy\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.227702 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.227684 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-podinfo\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.233335 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.233306 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-token\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.233435 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.233411 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz8fh\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-kube-api-access-hz8fh\") pod \"e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.389044 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.388966 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:01.532304 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.532275 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6"] Jul 03 21:12:01.533580 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:12:01.533558 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdaf55d2_0ac8_4fcf_b7ec_03a551b748cb.slice/crio-9c994eeb9be518360c5eac1c6d7cc32882022671dbe4188bdc6066f9f6f4fd20 WatchSource:0}: Error finding container 9c994eeb9be518360c5eac1c6d7cc32882022671dbe4188bdc6066f9f6f4fd20: Status 404 returned error can't find the container with id 9c994eeb9be518360c5eac1c6d7cc32882022671dbe4188bdc6066f9f6f4fd20 Jul 03 21:12:01.536051 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.536015 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:01.536156 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.536080 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:01.536156 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:01.536107 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:02.484965 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:02.484930 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" event={"ID":"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb","Type":"ContainerStarted","Data":"985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd"} Jul 03 21:12:02.485353 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:02.484990 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" event={"ID":"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb","Type":"ContainerStarted","Data":"9c994eeb9be518360c5eac1c6d7cc32882022671dbe4188bdc6066f9f6f4fd20"} Jul 03 21:12:02.506503 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:02.506456 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" podStartSLOduration=1.506441232 podStartE2EDuration="1.506441232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:12:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:12:02.503748689 +0000 UTC m=+2094.001316387" watchObservedRunningTime="2026-07-03 21:12:02.506441232 +0000 UTC m=+2094.004008892" Jul 03 21:12:03.389612 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:03.389575 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:03.394356 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:03.394330 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:03.490143 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:03.490105 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:03.491046 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:03.491031 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:04.718147 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.718119 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm"] Jul 03 21:12:04.718729 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.718712 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="06b6c0ab-d506-4fc4-8c9e-ad59aaf538f1" containerName="istio-proxy" Jul 03 21:12:04.721892 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.721869 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.724053 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.724030 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-a176cd-serving-cert\"" Jul 03 21:12:04.731810 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.731790 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm"] Jul 03 21:12:04.857478 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.857449 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg8sh\" (UniqueName: \"kubernetes.io/projected/c1667b2a-15d9-43db-9cc8-d57377c3dca9-kube-api-access-vg8sh\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.857629 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.857552 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1667b2a-15d9-43db-9cc8-d57377c3dca9-maas-api-tls\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.857629 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.857580 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1667b2a-15d9-43db-9cc8-d57377c3dca9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.958271 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.958235 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1667b2a-15d9-43db-9cc8-d57377c3dca9-maas-api-tls\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.958271 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.958273 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1667b2a-15d9-43db-9cc8-d57377c3dca9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.958527 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.958300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg8sh\" (UniqueName: \"kubernetes.io/projected/c1667b2a-15d9-43db-9cc8-d57377c3dca9-kube-api-access-vg8sh\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.958886 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.958865 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1667b2a-15d9-43db-9cc8-d57377c3dca9-openshift-service-ca\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.960904 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.960882 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1667b2a-15d9-43db-9cc8-d57377c3dca9-maas-api-tls\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:04.966145 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:04.966125 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg8sh\" (UniqueName: \"kubernetes.io/projected/c1667b2a-15d9-43db-9cc8-d57377c3dca9-kube-api-access-vg8sh\") pod \"maas-api-e2e-api-a-a176cd-858889c65-62scm\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:05.035725 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:05.035645 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:05.175477 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:05.175453 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm"] Jul 03 21:12:05.177265 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:12:05.177238 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1667b2a_15d9_43db_9cc8_d57377c3dca9.slice/crio-db4bddda16f89035fb40fe1f31e21ff5339ecda978c980ba4461852fd2d025a5 WatchSource:0}: Error finding container db4bddda16f89035fb40fe1f31e21ff5339ecda978c980ba4461852fd2d025a5: Status 404 returned error can't find the container with id db4bddda16f89035fb40fe1f31e21ff5339ecda978c980ba4461852fd2d025a5 Jul 03 21:12:05.178689 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:05.178666 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 21:12:05.502836 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:05.502800 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" event={"ID":"c1667b2a-15d9-43db-9cc8-d57377c3dca9","Type":"ContainerStarted","Data":"db4bddda16f89035fb40fe1f31e21ff5339ecda978c980ba4461852fd2d025a5"} Jul 03 21:12:06.509677 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:06.509644 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" event={"ID":"c1667b2a-15d9-43db-9cc8-d57377c3dca9","Type":"ContainerStarted","Data":"c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9"} Jul 03 21:12:06.510046 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:06.509802 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:06.526218 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:06.526170 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" podStartSLOduration=2.110473191 podStartE2EDuration="2.526157385s" totalImagesPullingTime="415.684194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:12:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:12:06.52472042 +0000 UTC m=+2098.022288080" watchObservedRunningTime="2026-07-03 21:12:06.526157385 +0000 UTC m=+2098.023725094" Jul 03 21:12:11.940864 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:11.940834 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:12:11.972784 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:11.972740 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:12:12.520839 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:12.520810 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:13.123781 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.122003 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl"] Jul 03 21:12:13.127719 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.127694 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.130432 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.130413 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-910755-serving-cert\"" Jul 03 21:12:13.139336 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.139272 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl"] Jul 03 21:12:13.234035 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.233954 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39f29997-ca93-46cb-aa0d-69f011647325-openshift-service-ca\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.234035 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.234008 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39f29997-ca93-46cb-aa0d-69f011647325-maas-api-tls\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.234254 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.234121 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g72k4\" (UniqueName: \"kubernetes.io/projected/39f29997-ca93-46cb-aa0d-69f011647325-kube-api-access-g72k4\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.335064 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.335035 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g72k4\" (UniqueName: \"kubernetes.io/projected/39f29997-ca93-46cb-aa0d-69f011647325-kube-api-access-g72k4\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.335254 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.335098 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39f29997-ca93-46cb-aa0d-69f011647325-openshift-service-ca\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.335254 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.335156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39f29997-ca93-46cb-aa0d-69f011647325-maas-api-tls\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.335877 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.335843 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39f29997-ca93-46cb-aa0d-69f011647325-openshift-service-ca\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.337717 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.337697 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39f29997-ca93-46cb-aa0d-69f011647325-maas-api-tls\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.343171 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.343144 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g72k4\" (UniqueName: \"kubernetes.io/projected/39f29997-ca93-46cb-aa0d-69f011647325-kube-api-access-g72k4\") pod \"maas-api-e2e-api-b-910755-8f6999bf5-dnskl\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.443634 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.443580 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:13.596785 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:13.596745 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl"] Jul 03 21:12:13.598785 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:12:13.598738 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f29997_ca93_46cb_aa0d_69f011647325.slice/crio-1b44913e006ed66bc1ea69501e49857ffb34cd0178e9c6f03e51bcbf1bc5fa8b WatchSource:0}: Error finding container 1b44913e006ed66bc1ea69501e49857ffb34cd0178e9c6f03e51bcbf1bc5fa8b: Status 404 returned error can't find the container with id 1b44913e006ed66bc1ea69501e49857ffb34cd0178e9c6f03e51bcbf1bc5fa8b Jul 03 21:12:14.560430 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:14.560396 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" event={"ID":"39f29997-ca93-46cb-aa0d-69f011647325","Type":"ContainerStarted","Data":"ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3"} Jul 03 21:12:14.560948 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:14.560446 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" event={"ID":"39f29997-ca93-46cb-aa0d-69f011647325","Type":"ContainerStarted","Data":"1b44913e006ed66bc1ea69501e49857ffb34cd0178e9c6f03e51bcbf1bc5fa8b"} Jul 03 21:12:14.560948 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:14.560505 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:14.576456 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:14.576410 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" podStartSLOduration=1.275725569 podStartE2EDuration="1.576395952s" totalImagesPullingTime="300.670383ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:12:14.574807027 +0000 UTC m=+2106.072374680" watchObservedRunningTime="2026-07-03 21:12:14.576395952 +0000 UTC m=+2106.073963612" Jul 03 21:12:16.571320 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:16.571283 2558 generic.go:356] "Generic (PLEG): container finished" podID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerID="985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd" exitCode=0 Jul 03 21:12:16.571851 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:16.571326 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" event={"ID":"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb","Type":"ContainerDied","Data":"985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd"} Jul 03 21:12:16.571851 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:16.571704 2558 scope.go:117] "RemoveContainer" containerID="985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd" Jul 03 21:12:16.572679 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:16.572649 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:16.572751 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:16.572720 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:16.572851 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:16.572750 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:17.578315 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:17.578281 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" event={"ID":"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb","Type":"ContainerStarted","Data":"61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f"} Jul 03 21:12:18.390106 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:18.390078 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:18.391209 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:18.391182 2558 patch_prober.go:28] interesting pod/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 03 21:12:18.391331 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:18.391237 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 03 21:12:18.490694 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:18.490663 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:19.389414 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:19.389382 2558 patch_prober.go:28] interesting pod/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 03 21:12:19.389821 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:19.389432 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 03 21:12:20.390388 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:20.390350 2558 patch_prober.go:28] interesting pod/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 03 21:12:20.390821 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:20.390404 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 03 21:12:20.574589 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:20.574558 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:21.393825 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:21.393799 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:21.394850 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:21.394832 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:26.126820 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.126753 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm"] Jul 03 21:12:26.127391 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.126998 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" podUID="c1667b2a-15d9-43db-9cc8-d57377c3dca9" containerName="maas-api" containerID="cri-o://c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9" gracePeriod=30 Jul 03 21:12:26.375287 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.375265 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:26.451997 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.451918 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1667b2a-15d9-43db-9cc8-d57377c3dca9-openshift-service-ca\") pod \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " Jul 03 21:12:26.451997 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.451992 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1667b2a-15d9-43db-9cc8-d57377c3dca9-maas-api-tls\") pod \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " Jul 03 21:12:26.452189 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.452063 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg8sh\" (UniqueName: \"kubernetes.io/projected/c1667b2a-15d9-43db-9cc8-d57377c3dca9-kube-api-access-vg8sh\") pod \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\" (UID: \"c1667b2a-15d9-43db-9cc8-d57377c3dca9\") " Jul 03 21:12:26.452259 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.452234 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1667b2a-15d9-43db-9cc8-d57377c3dca9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1667b2a-15d9-43db-9cc8-d57377c3dca9" (UID: "c1667b2a-15d9-43db-9cc8-d57377c3dca9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:12:26.452328 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.452316 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1667b2a-15d9-43db-9cc8-d57377c3dca9-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:26.454082 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.454052 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1667b2a-15d9-43db-9cc8-d57377c3dca9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1667b2a-15d9-43db-9cc8-d57377c3dca9" (UID: "c1667b2a-15d9-43db-9cc8-d57377c3dca9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:12:26.454201 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.454140 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1667b2a-15d9-43db-9cc8-d57377c3dca9-kube-api-access-vg8sh" (OuterVolumeSpecName: "kube-api-access-vg8sh") pod "c1667b2a-15d9-43db-9cc8-d57377c3dca9" (UID: "c1667b2a-15d9-43db-9cc8-d57377c3dca9"). InnerVolumeSpecName "kube-api-access-vg8sh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:12:26.552873 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.552845 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1667b2a-15d9-43db-9cc8-d57377c3dca9-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:26.552873 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.552869 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg8sh\" (UniqueName: \"kubernetes.io/projected/c1667b2a-15d9-43db-9cc8-d57377c3dca9-kube-api-access-vg8sh\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:26.629613 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.629578 2558 generic.go:356] "Generic (PLEG): container finished" podID="c1667b2a-15d9-43db-9cc8-d57377c3dca9" containerID="c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9" exitCode=0 Jul 03 21:12:26.629810 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.629620 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" Jul 03 21:12:26.629810 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.629671 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" event={"ID":"c1667b2a-15d9-43db-9cc8-d57377c3dca9","Type":"ContainerDied","Data":"c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9"} Jul 03 21:12:26.629810 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.629730 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm" event={"ID":"c1667b2a-15d9-43db-9cc8-d57377c3dca9","Type":"ContainerDied","Data":"db4bddda16f89035fb40fe1f31e21ff5339ecda978c980ba4461852fd2d025a5"} Jul 03 21:12:26.629810 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.629784 2558 scope.go:117] "RemoveContainer" containerID="c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9" Jul 03 21:12:26.641519 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.641493 2558 scope.go:117] "RemoveContainer" containerID="c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9" Jul 03 21:12:26.642137 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:12:26.642115 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9\": container with ID starting with c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9 not found: ID does not exist" containerID="c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9" Jul 03 21:12:26.642229 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.642149 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9"} err="failed to get container status \"c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9\": rpc error: code = NotFound desc = could not find container \"c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9\": container with ID starting with c4c22bcbc3535a453486dfbe4b598bd9107d465d389bac9ef854c6d4b4ffe2b9 not found: ID does not exist" Jul 03 21:12:26.654623 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.654601 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm"] Jul 03 21:12:26.658057 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:26.658038 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a176cd-858889c65-62scm"] Jul 03 21:12:27.052350 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:27.052318 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1667b2a-15d9-43db-9cc8-d57377c3dca9" path="/var/lib/kubelet/pods/c1667b2a-15d9-43db-9cc8-d57377c3dca9/volumes" Jul 03 21:12:32.001080 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:32.001047 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6"] Jul 03 21:12:32.001528 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:32.001269 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" containerID="cri-o://61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f" gracePeriod=30 Jul 03 21:12:32.979532 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:32.979489 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl"] Jul 03 21:12:32.979861 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:32.979822 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" podUID="39f29997-ca93-46cb-aa0d-69f011647325" containerName="maas-api" containerID="cri-o://ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3" gracePeriod=30 Jul 03 21:12:33.226671 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.226644 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:33.408118 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.408084 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39f29997-ca93-46cb-aa0d-69f011647325-openshift-service-ca\") pod \"39f29997-ca93-46cb-aa0d-69f011647325\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " Jul 03 21:12:33.408118 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.408123 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39f29997-ca93-46cb-aa0d-69f011647325-maas-api-tls\") pod \"39f29997-ca93-46cb-aa0d-69f011647325\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " Jul 03 21:12:33.408386 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.408184 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g72k4\" (UniqueName: \"kubernetes.io/projected/39f29997-ca93-46cb-aa0d-69f011647325-kube-api-access-g72k4\") pod \"39f29997-ca93-46cb-aa0d-69f011647325\" (UID: \"39f29997-ca93-46cb-aa0d-69f011647325\") " Jul 03 21:12:33.408535 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.408509 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39f29997-ca93-46cb-aa0d-69f011647325-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39f29997-ca93-46cb-aa0d-69f011647325" (UID: "39f29997-ca93-46cb-aa0d-69f011647325"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:12:33.410300 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.410278 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39f29997-ca93-46cb-aa0d-69f011647325-kube-api-access-g72k4" (OuterVolumeSpecName: "kube-api-access-g72k4") pod "39f29997-ca93-46cb-aa0d-69f011647325" (UID: "39f29997-ca93-46cb-aa0d-69f011647325"). InnerVolumeSpecName "kube-api-access-g72k4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:12:33.410388 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.410327 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39f29997-ca93-46cb-aa0d-69f011647325-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39f29997-ca93-46cb-aa0d-69f011647325" (UID: "39f29997-ca93-46cb-aa0d-69f011647325"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:12:33.509190 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.509154 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39f29997-ca93-46cb-aa0d-69f011647325-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:33.509190 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.509184 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g72k4\" (UniqueName: \"kubernetes.io/projected/39f29997-ca93-46cb-aa0d-69f011647325-kube-api-access-g72k4\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:33.509190 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.509193 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39f29997-ca93-46cb-aa0d-69f011647325-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:33.668356 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.668265 2558 generic.go:356] "Generic (PLEG): container finished" podID="39f29997-ca93-46cb-aa0d-69f011647325" containerID="ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3" exitCode=0 Jul 03 21:12:33.668356 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.668311 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" Jul 03 21:12:33.668544 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.668353 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" event={"ID":"39f29997-ca93-46cb-aa0d-69f011647325","Type":"ContainerDied","Data":"ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3"} Jul 03 21:12:33.668544 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.668406 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl" event={"ID":"39f29997-ca93-46cb-aa0d-69f011647325","Type":"ContainerDied","Data":"1b44913e006ed66bc1ea69501e49857ffb34cd0178e9c6f03e51bcbf1bc5fa8b"} Jul 03 21:12:33.668544 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.668428 2558 scope.go:117] "RemoveContainer" containerID="ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3" Jul 03 21:12:33.677838 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.677818 2558 scope.go:117] "RemoveContainer" containerID="ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3" Jul 03 21:12:33.678088 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:12:33.678069 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3\": container with ID starting with ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3 not found: ID does not exist" containerID="ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3" Jul 03 21:12:33.678161 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.678095 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3"} err="failed to get container status \"ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3\": rpc error: code = NotFound desc = could not find container \"ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3\": container with ID starting with ed142a90831cd56d4788234c8ed0b19701176c5724353c2dad68f94b454922c3 not found: ID does not exist" Jul 03 21:12:33.690032 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.690013 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl"] Jul 03 21:12:33.693865 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:33.693846 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-910755-8f6999bf5-dnskl"] Jul 03 21:12:35.051677 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:35.051643 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39f29997-ca93-46cb-aa0d-69f011647325" path="/var/lib/kubelet/pods/39f29997-ca93-46cb-aa0d-69f011647325/volumes" Jul 03 21:12:36.396899 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:36.396859 2558 patch_prober.go:28] interesting pod/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 03 21:12:36.397269 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:36.396926 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 03 21:12:37.253963 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.253934 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:37.446489 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446453 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz8fh\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-kube-api-access-hz8fh\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.446489 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446488 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-envoy\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446508 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istiod-ca-cert\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446559 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-certs\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446586 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-data\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446617 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-podinfo\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446649 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-socket\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446678 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-credential-socket\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446829 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-token\") pod \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\" (UID: \"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb\") " Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.446987 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:12:37.447099 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.447006 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:12:37.447597 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.447220 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-certs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.447597 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.447263 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-workload-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.447597 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.447366 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:12:37.447597 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.447423 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:12:37.447843 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.447664 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:12:37.448920 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.448886 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:12:37.448920 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.448890 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 21:12:37.448920 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.448892 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-kube-api-access-hz8fh" (OuterVolumeSpecName: "kube-api-access-hz8fh") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "kube-api-access-hz8fh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:12:37.449160 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.448969 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" (UID: "cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:12:37.547672 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.547642 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istiod-ca-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.547672 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.547665 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-data\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.547672 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.547674 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-podinfo\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.547904 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.547682 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-credential-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.547904 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.547692 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-token\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.547904 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.547700 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz8fh\" (UniqueName: \"kubernetes.io/projected/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-kube-api-access-hz8fh\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.547904 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.547708 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb-istio-envoy\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:12:37.691169 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.691134 2558 generic.go:356] "Generic (PLEG): container finished" podID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerID="61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f" exitCode=0 Jul 03 21:12:37.691331 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.691178 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" Jul 03 21:12:37.691331 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.691219 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" event={"ID":"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb","Type":"ContainerDied","Data":"61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f"} Jul 03 21:12:37.691331 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.691278 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6" event={"ID":"cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb","Type":"ContainerDied","Data":"9c994eeb9be518360c5eac1c6d7cc32882022671dbe4188bdc6066f9f6f4fd20"} Jul 03 21:12:37.691331 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.691304 2558 scope.go:117] "RemoveContainer" containerID="61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f" Jul 03 21:12:37.707917 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.707898 2558 scope.go:117] "RemoveContainer" containerID="985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd" Jul 03 21:12:37.717016 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.716999 2558 scope.go:117] "RemoveContainer" containerID="61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f" Jul 03 21:12:37.717251 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:12:37.717233 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f\": container with ID starting with 61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f not found: ID does not exist" containerID="61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f" Jul 03 21:12:37.717303 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.717258 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f"} err="failed to get container status \"61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f\": rpc error: code = NotFound desc = could not find container \"61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f\": container with ID starting with 61724fb30086428e00e7bf7cdd05dadea2ed774c00a2a29dfd0ea6833a255e0f not found: ID does not exist" Jul 03 21:12:37.717303 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.717273 2558 scope.go:117] "RemoveContainer" containerID="985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd" Jul 03 21:12:37.717499 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:12:37.717481 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd\": container with ID starting with 985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd not found: ID does not exist" containerID="985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd" Jul 03 21:12:37.717605 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.717508 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd"} err="failed to get container status \"985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd\": rpc error: code = NotFound desc = could not find container \"985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd\": container with ID starting with 985e9698c20c634be48c04d2078a07736bae10a9caf0012174bb1120496fb2dd not found: ID does not exist" Jul 03 21:12:37.717913 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.717865 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6"] Jul 03 21:12:37.720558 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:37.720539 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a176cd-openshift-default-6c79f4d6ff-k6cv6"] Jul 03 21:12:39.052714 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.052682 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" path="/var/lib/kubelet/pods/cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb/volumes" Jul 03 21:12:39.677720 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.677692 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr"] Jul 03 21:12:39.678065 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678052 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" Jul 03 21:12:39.678108 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678067 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" Jul 03 21:12:39.678108 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678079 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39f29997-ca93-46cb-aa0d-69f011647325" containerName="maas-api" Jul 03 21:12:39.678108 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678084 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="39f29997-ca93-46cb-aa0d-69f011647325" containerName="maas-api" Jul 03 21:12:39.678108 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678092 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1667b2a-15d9-43db-9cc8-d57377c3dca9" containerName="maas-api" Jul 03 21:12:39.678108 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678097 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1667b2a-15d9-43db-9cc8-d57377c3dca9" containerName="maas-api" Jul 03 21:12:39.678273 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678166 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" Jul 03 21:12:39.678273 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678174 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" Jul 03 21:12:39.678273 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678179 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1667b2a-15d9-43db-9cc8-d57377c3dca9" containerName="maas-api" Jul 03 21:12:39.678273 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.678186 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="39f29997-ca93-46cb-aa0d-69f011647325" containerName="maas-api" Jul 03 21:12:39.682528 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.682511 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.685091 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.685069 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1a4753-openshift-default-dockercfg-ztmxd\"" Jul 03 21:12:39.695451 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.695427 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr"] Jul 03 21:12:39.867652 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-data\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.867848 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867659 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-token\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.867848 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867684 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr9xn\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-kube-api-access-jr9xn\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.867848 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867706 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-envoy\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.867848 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867726 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-podinfo\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.867848 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867810 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istiod-ca-cert\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.867848 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867834 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-socket\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.868103 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867853 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-credential-socket\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.868103 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.867868 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-certs\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969275 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969203 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-data\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969275 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969237 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-token\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969275 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969255 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr9xn\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-kube-api-access-jr9xn\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969280 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-envoy\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969308 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-podinfo\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969361 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istiod-ca-cert\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-socket\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969421 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-credential-socket\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969534 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969442 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-certs\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969835 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969580 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-data\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969835 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969822 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-certs\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969928 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969873 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-socket\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.969995 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.969973 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-credential-socket\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.970079 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.970062 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istiod-ca-cert\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.971711 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.971691 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-envoy\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.971885 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.971870 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-podinfo\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.976920 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.976898 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-token\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.977121 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.977104 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr9xn\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-kube-api-access-jr9xn\") pod \"e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:39.995911 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:39.995888 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:40.130160 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.130127 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr"] Jul 03 21:12:40.138591 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:12:40.138563 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc7640c_804e_4243_82ee_ed172ac5fbcc.slice/crio-f2de37c388256d86ff7158c4d47d37884a67e690547e6de3b7a808ba122ffa53 WatchSource:0}: Error finding container f2de37c388256d86ff7158c4d47d37884a67e690547e6de3b7a808ba122ffa53: Status 404 returned error can't find the container with id f2de37c388256d86ff7158c4d47d37884a67e690547e6de3b7a808ba122ffa53 Jul 03 21:12:40.140865 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.140833 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:40.140964 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.140901 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:40.140964 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.140927 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:40.711138 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.711104 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" event={"ID":"bdc7640c-804e-4243-82ee-ed172ac5fbcc","Type":"ContainerStarted","Data":"ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf"} Jul 03 21:12:40.711319 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.711156 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" event={"ID":"bdc7640c-804e-4243-82ee-ed172ac5fbcc","Type":"ContainerStarted","Data":"f2de37c388256d86ff7158c4d47d37884a67e690547e6de3b7a808ba122ffa53"} Jul 03 21:12:40.732196 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.732151 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" podStartSLOduration=1.732134675 podStartE2EDuration="1.732134675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:12:40.729275601 +0000 UTC m=+2132.226843265" watchObservedRunningTime="2026-07-03 21:12:40.732134675 +0000 UTC m=+2132.229702334" Jul 03 21:12:40.996667 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:40.996578 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:42.001702 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.001674 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:42.725161 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.725130 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:42.726020 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.726003 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:42.745186 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.745145 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm"] Jul 03 21:12:42.745687 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.745664 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" Jul 03 21:12:42.745687 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.745686 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdaf55d2-0ac8-4fcf-b7ec-03a551b748cb" containerName="istio-proxy" Jul 03 21:12:42.749787 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.749287 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.751816 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.751791 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-1a4753-serving-cert\"" Jul 03 21:12:42.764457 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.764433 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm"] Jul 03 21:12:42.791653 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.791615 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt6r7\" (UniqueName: \"kubernetes.io/projected/a6078d95-e0be-41e9-a576-262023435ddc-kube-api-access-nt6r7\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.791872 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.791669 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6078d95-e0be-41e9-a576-262023435ddc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.791872 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.791712 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6078d95-e0be-41e9-a576-262023435ddc-maas-api-tls\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.892787 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.892731 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt6r7\" (UniqueName: \"kubernetes.io/projected/a6078d95-e0be-41e9-a576-262023435ddc-kube-api-access-nt6r7\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.892968 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.892817 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6078d95-e0be-41e9-a576-262023435ddc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.892968 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.892848 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6078d95-e0be-41e9-a576-262023435ddc-maas-api-tls\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.893515 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.893486 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6078d95-e0be-41e9-a576-262023435ddc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.895375 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.895356 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6078d95-e0be-41e9-a576-262023435ddc-maas-api-tls\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:42.907209 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:42.907182 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt6r7\" (UniqueName: \"kubernetes.io/projected/a6078d95-e0be-41e9-a576-262023435ddc-kube-api-access-nt6r7\") pod \"maas-api-e2e-shared-a-1a4753-68589b7645-czctm\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:43.068628 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:43.068603 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:43.206183 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:43.206158 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm"] Jul 03 21:12:43.207523 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:12:43.207497 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6078d95_e0be_41e9_a576_262023435ddc.slice/crio-656df5abc0238230cf1e4b07ced3b798efce68cee72f1931f0ef7c7bf3fff81a WatchSource:0}: Error finding container 656df5abc0238230cf1e4b07ced3b798efce68cee72f1931f0ef7c7bf3fff81a: Status 404 returned error can't find the container with id 656df5abc0238230cf1e4b07ced3b798efce68cee72f1931f0ef7c7bf3fff81a Jul 03 21:12:43.732438 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:43.732403 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" event={"ID":"a6078d95-e0be-41e9-a576-262023435ddc","Type":"ContainerStarted","Data":"2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9"} Jul 03 21:12:43.732637 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:43.732452 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" event={"ID":"a6078d95-e0be-41e9-a576-262023435ddc","Type":"ContainerStarted","Data":"656df5abc0238230cf1e4b07ced3b798efce68cee72f1931f0ef7c7bf3fff81a"} Jul 03 21:12:43.732637 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:43.732480 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:43.748593 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:43.748538 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" podStartSLOduration=1.3408835080000001 podStartE2EDuration="1.748520592s" totalImagesPullingTime="407.637084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:12:43.747015651 +0000 UTC m=+2135.244583312" watchObservedRunningTime="2026-07-03 21:12:43.748520592 +0000 UTC m=+2135.246088252" Jul 03 21:12:49.743724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:49.743695 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:12:54.803031 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:54.802998 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerID="ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf" exitCode=0 Jul 03 21:12:54.803445 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:54.803046 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" event={"ID":"bdc7640c-804e-4243-82ee-ed172ac5fbcc","Type":"ContainerDied","Data":"ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf"} Jul 03 21:12:54.803445 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:54.803379 2558 scope.go:117] "RemoveContainer" containerID="ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf" Jul 03 21:12:54.804435 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:54.804408 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:54.804496 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:54.804481 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:54.804547 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:54.804515 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 03 21:12:55.809844 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:55.809811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" event={"ID":"bdc7640c-804e-4243-82ee-ed172ac5fbcc","Type":"ContainerStarted","Data":"bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac"} Jul 03 21:12:55.996451 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:55.996424 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:55.997621 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:55.997591 2558 patch_prober.go:28] interesting pod/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 03 21:12:55.997797 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:55.997639 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 03 21:12:56.997092 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:56.997056 2558 patch_prober.go:28] interesting pod/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 03 21:12:56.997586 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:56.997118 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 03 21:12:57.725188 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:57.725157 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:12:57.996860 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:57.996739 2558 patch_prober.go:28] interesting pod/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 03 21:12:57.996860 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:57.996829 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 03 21:12:58.996635 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:58.996585 2558 patch_prober.go:28] interesting pod/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 03 21:12:58.997134 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:12:58.996681 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 03 21:13:00.000778 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:13:00.000730 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:13:00.001683 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:13:00.001662 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:15:00.132145 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.132108 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs"] Jul 03 21:15:00.135553 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.135529 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62"] Jul 03 21:15:00.135738 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.135663 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" Jul 03 21:15:00.139518 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.139500 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29718555-cspwh"] Jul 03 21:15:00.139655 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.139622 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" Jul 03 21:15:00.143982 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.143258 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" Jul 03 21:15:00.145625 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.145599 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718555-cspwh"] Jul 03 21:15:00.148163 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.148144 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs"] Jul 03 21:15:00.158838 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.158816 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62"] Jul 03 21:15:00.182789 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.182746 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7przb\" (UniqueName: \"kubernetes.io/projected/2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a-kube-api-access-7przb\") pod \"maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62\" (UID: \"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" Jul 03 21:15:00.183019 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.183000 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn5mq\" (UniqueName: \"kubernetes.io/projected/a10dd446-9bf9-4ba3-85f0-668ea6754de0-kube-api-access-xn5mq\") pod \"maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs\" (UID: \"a10dd446-9bf9-4ba3-85f0-668ea6754de0\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" Jul 03 21:15:00.283551 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.283520 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7przb\" (UniqueName: \"kubernetes.io/projected/2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a-kube-api-access-7przb\") pod \"maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62\" (UID: \"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" Jul 03 21:15:00.283731 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.283594 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn5mq\" (UniqueName: \"kubernetes.io/projected/a10dd446-9bf9-4ba3-85f0-668ea6754de0-kube-api-access-xn5mq\") pod \"maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs\" (UID: \"a10dd446-9bf9-4ba3-85f0-668ea6754de0\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" Jul 03 21:15:00.283731 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.283686 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttg4t\" (UniqueName: \"kubernetes.io/projected/40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b-kube-api-access-ttg4t\") pod \"maas-api-key-cleanup-29718555-cspwh\" (UID: \"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b\") " pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" Jul 03 21:15:00.291876 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.291842 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7przb\" (UniqueName: \"kubernetes.io/projected/2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a-kube-api-access-7przb\") pod \"maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62\" (UID: \"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" Jul 03 21:15:00.291876 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.291867 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn5mq\" (UniqueName: \"kubernetes.io/projected/a10dd446-9bf9-4ba3-85f0-668ea6754de0-kube-api-access-xn5mq\") pod \"maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs\" (UID: \"a10dd446-9bf9-4ba3-85f0-668ea6754de0\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" Jul 03 21:15:00.385100 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.385015 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttg4t\" (UniqueName: \"kubernetes.io/projected/40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b-kube-api-access-ttg4t\") pod \"maas-api-key-cleanup-29718555-cspwh\" (UID: \"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b\") " pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" Jul 03 21:15:00.392893 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.392858 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttg4t\" (UniqueName: \"kubernetes.io/projected/40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b-kube-api-access-ttg4t\") pod \"maas-api-key-cleanup-29718555-cspwh\" (UID: \"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b\") " pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" Jul 03 21:15:00.449144 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.449123 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" Jul 03 21:15:00.458260 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.458236 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" Jul 03 21:15:00.465878 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.465856 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" Jul 03 21:15:00.602235 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.602207 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs"] Jul 03 21:15:00.830460 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.830437 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62"] Jul 03 21:15:00.831351 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:15:00.831326 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e4bb2dc_545d_49a2_b58c_c6c0cd59a54a.slice/crio-f4002d41db6581893fed5af4d3ef08a62a6e172efc897741f1d328a47278a4b1 WatchSource:0}: Error finding container f4002d41db6581893fed5af4d3ef08a62a6e172efc897741f1d328a47278a4b1: Status 404 returned error can't find the container with id f4002d41db6581893fed5af4d3ef08a62a6e172efc897741f1d328a47278a4b1 Jul 03 21:15:00.853548 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:00.853528 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29718555-cspwh"] Jul 03 21:15:00.860694 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:15:00.860670 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b901b9_683a_4c8d_8bf5_6e1bb2b96f5b.slice/crio-dacba75865fd4e3b2b85f872201eb25d4697e48e777905edc8fca596fc57c721 WatchSource:0}: Error finding container dacba75865fd4e3b2b85f872201eb25d4697e48e777905edc8fca596fc57c721: Status 404 returned error can't find the container with id dacba75865fd4e3b2b85f872201eb25d4697e48e777905edc8fca596fc57c721 Jul 03 21:15:01.490555 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.490520 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" event={"ID":"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b","Type":"ContainerStarted","Data":"b3d114caa667c098e24e1237651bfeeaa6e3e777fb19a5e289de07e270061aa7"} Jul 03 21:15:01.490977 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.490580 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" event={"ID":"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b","Type":"ContainerStarted","Data":"dacba75865fd4e3b2b85f872201eb25d4697e48e777905edc8fca596fc57c721"} Jul 03 21:15:01.492115 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.492094 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" event={"ID":"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a","Type":"ContainerStarted","Data":"905bf285ad3cfafb05a175187692e0b847a741a95ddc4d3cfab1ac4d15d3e519"} Jul 03 21:15:01.492253 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.492140 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" event={"ID":"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a","Type":"ContainerStarted","Data":"f4002d41db6581893fed5af4d3ef08a62a6e172efc897741f1d328a47278a4b1"} Jul 03 21:15:01.493434 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.493411 2558 generic.go:356] "Generic (PLEG): container finished" podID="a10dd446-9bf9-4ba3-85f0-668ea6754de0" containerID="501eca4e5eba0c1c4743a94f64d7c67c4d7807e644925d34e8a4e23d7e3ee831" exitCode=0 Jul 03 21:15:01.493523 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.493489 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" event={"ID":"a10dd446-9bf9-4ba3-85f0-668ea6754de0","Type":"ContainerDied","Data":"501eca4e5eba0c1c4743a94f64d7c67c4d7807e644925d34e8a4e23d7e3ee831"} Jul 03 21:15:01.493572 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.493525 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" event={"ID":"a10dd446-9bf9-4ba3-85f0-668ea6754de0","Type":"ContainerStarted","Data":"1da45cacb46c2c321991595a22d8bf483dd123ee7fc5baeaa96929ec66f02dbe"} Jul 03 21:15:01.505258 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.505220 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" podStartSLOduration=1.505205428 podStartE2EDuration="1.505205428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:15:01.504057314 +0000 UTC m=+2273.001624991" watchObservedRunningTime="2026-07-03 21:15:01.505205428 +0000 UTC m=+2273.002773085" Jul 03 21:15:01.533368 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:01.533327 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" podStartSLOduration=1.533310631 podStartE2EDuration="1.533310631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:15:01.532287793 +0000 UTC m=+2273.029855455" watchObservedRunningTime="2026-07-03 21:15:01.533310631 +0000 UTC m=+2273.030878291" Jul 03 21:15:02.500131 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.500094 2558 generic.go:356] "Generic (PLEG): container finished" podID="40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b" containerID="b3d114caa667c098e24e1237651bfeeaa6e3e777fb19a5e289de07e270061aa7" exitCode=0 Jul 03 21:15:02.500535 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.500158 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" event={"ID":"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b","Type":"ContainerDied","Data":"b3d114caa667c098e24e1237651bfeeaa6e3e777fb19a5e289de07e270061aa7"} Jul 03 21:15:02.501647 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.501630 2558 generic.go:356] "Generic (PLEG): container finished" podID="2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a" containerID="905bf285ad3cfafb05a175187692e0b847a741a95ddc4d3cfab1ac4d15d3e519" exitCode=0 Jul 03 21:15:02.501724 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.501706 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" event={"ID":"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a","Type":"ContainerDied","Data":"905bf285ad3cfafb05a175187692e0b847a741a95ddc4d3cfab1ac4d15d3e519"} Jul 03 21:15:02.632088 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.632066 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" Jul 03 21:15:02.704872 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.704846 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn5mq\" (UniqueName: \"kubernetes.io/projected/a10dd446-9bf9-4ba3-85f0-668ea6754de0-kube-api-access-xn5mq\") pod \"a10dd446-9bf9-4ba3-85f0-668ea6754de0\" (UID: \"a10dd446-9bf9-4ba3-85f0-668ea6754de0\") " Jul 03 21:15:02.706892 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.706870 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10dd446-9bf9-4ba3-85f0-668ea6754de0-kube-api-access-xn5mq" (OuterVolumeSpecName: "kube-api-access-xn5mq") pod "a10dd446-9bf9-4ba3-85f0-668ea6754de0" (UID: "a10dd446-9bf9-4ba3-85f0-668ea6754de0"). InnerVolumeSpecName "kube-api-access-xn5mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:15:02.806332 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:02.806305 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn5mq\" (UniqueName: \"kubernetes.io/projected/a10dd446-9bf9-4ba3-85f0-668ea6754de0-kube-api-access-xn5mq\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:15:03.508089 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.508009 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" Jul 03 21:15:03.508089 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.508054 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs" event={"ID":"a10dd446-9bf9-4ba3-85f0-668ea6754de0","Type":"ContainerDied","Data":"1da45cacb46c2c321991595a22d8bf483dd123ee7fc5baeaa96929ec66f02dbe"} Jul 03 21:15:03.508538 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.508138 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1da45cacb46c2c321991595a22d8bf483dd123ee7fc5baeaa96929ec66f02dbe" Jul 03 21:15:03.644754 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.644733 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" Jul 03 21:15:03.666647 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.666627 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" Jul 03 21:15:03.715687 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.715658 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7przb\" (UniqueName: \"kubernetes.io/projected/2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a-kube-api-access-7przb\") pod \"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a\" (UID: \"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a\") " Jul 03 21:15:03.717537 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.717512 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a-kube-api-access-7przb" (OuterVolumeSpecName: "kube-api-access-7przb") pod "2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a" (UID: "2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a"). InnerVolumeSpecName "kube-api-access-7przb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:15:03.817237 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.817209 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttg4t\" (UniqueName: \"kubernetes.io/projected/40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b-kube-api-access-ttg4t\") pod \"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b\" (UID: \"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b\") " Jul 03 21:15:03.817414 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.817402 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7przb\" (UniqueName: \"kubernetes.io/projected/2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a-kube-api-access-7przb\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:15:03.819133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.819105 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b-kube-api-access-ttg4t" (OuterVolumeSpecName: "kube-api-access-ttg4t") pod "40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b" (UID: "40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b"). InnerVolumeSpecName "kube-api-access-ttg4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:15:03.918580 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:03.918555 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttg4t\" (UniqueName: \"kubernetes.io/projected/40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b-kube-api-access-ttg4t\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:15:04.514586 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:04.514557 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" Jul 03 21:15:04.515014 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:04.514601 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62" event={"ID":"2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a","Type":"ContainerDied","Data":"f4002d41db6581893fed5af4d3ef08a62a6e172efc897741f1d328a47278a4b1"} Jul 03 21:15:04.515014 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:04.514653 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4002d41db6581893fed5af4d3ef08a62a6e172efc897741f1d328a47278a4b1" Jul 03 21:15:04.515980 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:04.515958 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" event={"ID":"40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b","Type":"ContainerDied","Data":"dacba75865fd4e3b2b85f872201eb25d4697e48e777905edc8fca596fc57c721"} Jul 03 21:15:04.516091 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:04.515992 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dacba75865fd4e3b2b85f872201eb25d4697e48e777905edc8fca596fc57c721" Jul 03 21:15:04.516091 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:15:04.516031 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29718555-cspwh" Jul 03 21:17:12.006042 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:17:12.006007 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:17:12.014253 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:17:12.014230 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:18:10.194481 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.194446 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm"] Jul 03 21:18:10.197223 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.194679 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" podUID="a6078d95-e0be-41e9-a576-262023435ddc" containerName="maas-api" containerID="cri-o://2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9" gracePeriod=30 Jul 03 21:18:10.218171 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.218145 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62"] Jul 03 21:18:10.220405 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.220386 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-1a4753-29718555-wkm62"] Jul 03 21:18:10.442481 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.442456 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:18:10.548371 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.548334 2558 generic.go:356] "Generic (PLEG): container finished" podID="a6078d95-e0be-41e9-a576-262023435ddc" containerID="2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9" exitCode=0 Jul 03 21:18:10.548570 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.548378 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" Jul 03 21:18:10.548570 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.548421 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" event={"ID":"a6078d95-e0be-41e9-a576-262023435ddc","Type":"ContainerDied","Data":"2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9"} Jul 03 21:18:10.548570 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.548481 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm" event={"ID":"a6078d95-e0be-41e9-a576-262023435ddc","Type":"ContainerDied","Data":"656df5abc0238230cf1e4b07ced3b798efce68cee72f1931f0ef7c7bf3fff81a"} Jul 03 21:18:10.548570 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.548511 2558 scope.go:117] "RemoveContainer" containerID="2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9" Jul 03 21:18:10.557276 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.557260 2558 scope.go:117] "RemoveContainer" containerID="2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9" Jul 03 21:18:10.557539 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:18:10.557519 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9\": container with ID starting with 2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9 not found: ID does not exist" containerID="2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9" Jul 03 21:18:10.557614 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.557550 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9"} err="failed to get container status \"2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9\": rpc error: code = NotFound desc = could not find container \"2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9\": container with ID starting with 2c17bd065008e72fd4ac7047426b6e6b332c0f1f9f4d7d138e3d4df4fc4f9bf9 not found: ID does not exist" Jul 03 21:18:10.610454 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.610420 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nt6r7\" (UniqueName: \"kubernetes.io/projected/a6078d95-e0be-41e9-a576-262023435ddc-kube-api-access-nt6r7\") pod \"a6078d95-e0be-41e9-a576-262023435ddc\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " Jul 03 21:18:10.610619 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.610608 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6078d95-e0be-41e9-a576-262023435ddc-maas-api-tls\") pod \"a6078d95-e0be-41e9-a576-262023435ddc\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " Jul 03 21:18:10.610693 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.610641 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6078d95-e0be-41e9-a576-262023435ddc-openshift-service-ca\") pod \"a6078d95-e0be-41e9-a576-262023435ddc\" (UID: \"a6078d95-e0be-41e9-a576-262023435ddc\") " Jul 03 21:18:10.611052 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.611024 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6078d95-e0be-41e9-a576-262023435ddc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6078d95-e0be-41e9-a576-262023435ddc" (UID: "a6078d95-e0be-41e9-a576-262023435ddc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:18:10.612578 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.612554 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6078d95-e0be-41e9-a576-262023435ddc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6078d95-e0be-41e9-a576-262023435ddc" (UID: "a6078d95-e0be-41e9-a576-262023435ddc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 21:18:10.612578 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.612566 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6078d95-e0be-41e9-a576-262023435ddc-kube-api-access-nt6r7" (OuterVolumeSpecName: "kube-api-access-nt6r7") pod "a6078d95-e0be-41e9-a576-262023435ddc" (UID: "a6078d95-e0be-41e9-a576-262023435ddc"). InnerVolumeSpecName "kube-api-access-nt6r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:18:10.712201 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.712172 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6078d95-e0be-41e9-a576-262023435ddc-maas-api-tls\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:10.712201 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.712195 2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6078d95-e0be-41e9-a576-262023435ddc-openshift-service-ca\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:10.712201 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.712204 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nt6r7\" (UniqueName: \"kubernetes.io/projected/a6078d95-e0be-41e9-a576-262023435ddc-kube-api-access-nt6r7\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:10.876153 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.870959 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm"] Jul 03 21:18:10.879427 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:10.879389 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-1a4753-68589b7645-czctm"] Jul 03 21:18:11.051583 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:11.051540 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a" path="/var/lib/kubelet/pods/2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a/volumes" Jul 03 21:18:11.051997 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:11.051983 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6078d95-e0be-41e9-a576-262023435ddc" path="/var/lib/kubelet/pods/a6078d95-e0be-41e9-a576-262023435ddc/volumes" Jul 03 21:18:52.409621 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:52.409587 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr"] Jul 03 21:18:52.410392 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:52.409873 2558 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" containerID="cri-o://bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac" gracePeriod=30 Jul 03 21:18:53.318552 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:53.318516 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs"] Jul 03 21:18:53.323493 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:53.323458 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-26641f-29718555-7zhjs"] Jul 03 21:18:55.051683 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:55.051650 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a10dd446-9bf9-4ba3-85f0-668ea6754de0" path="/var/lib/kubelet/pods/a10dd446-9bf9-4ba3-85f0-668ea6754de0/volumes" Jul 03 21:18:57.664957 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.664928 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:18:57.807639 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.807611 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerID="bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac" exitCode=0 Jul 03 21:18:57.807823 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.807665 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" Jul 03 21:18:57.807823 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.807695 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" event={"ID":"bdc7640c-804e-4243-82ee-ed172ac5fbcc","Type":"ContainerDied","Data":"bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac"} Jul 03 21:18:57.807823 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.807756 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr" event={"ID":"bdc7640c-804e-4243-82ee-ed172ac5fbcc","Type":"ContainerDied","Data":"f2de37c388256d86ff7158c4d47d37884a67e690547e6de3b7a808ba122ffa53"} Jul 03 21:18:57.807823 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.807786 2558 scope.go:117] "RemoveContainer" containerID="bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac" Jul 03 21:18:57.818524 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.818505 2558 scope.go:117] "RemoveContainer" containerID="ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf" Jul 03 21:18:57.823902 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.823883 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-token\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.823998 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.823926 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-data\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.823998 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.823951 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-socket\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.823998 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.823990 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr9xn\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-kube-api-access-jr9xn\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.824133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824007 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-podinfo\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.824133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824042 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-envoy\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.824133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824066 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-credential-socket\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.824133 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824092 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-certs\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.824345 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824155 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istiod-ca-cert\") pod \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\" (UID: \"bdc7640c-804e-4243-82ee-ed172ac5fbcc\") " Jul 03 21:18:57.824655 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824601 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:18:57.824744 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824670 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:18:57.824744 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824695 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 21:18:57.824886 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824784 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-data" (OuterVolumeSpecName: "istio-data") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:18:57.824949 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.824890 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:18:57.826157 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.826129 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-token" (OuterVolumeSpecName: "istio-token") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:18:57.826253 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.826234 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 03 21:18:57.826295 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.826239 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-kube-api-access-jr9xn" (OuterVolumeSpecName: "kube-api-access-jr9xn") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "kube-api-access-jr9xn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 21:18:57.826950 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.826926 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bdc7640c-804e-4243-82ee-ed172ac5fbcc" (UID: "bdc7640c-804e-4243-82ee-ed172ac5fbcc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 21:18:57.827739 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.827724 2558 scope.go:117] "RemoveContainer" containerID="bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac" Jul 03 21:18:57.828033 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:18:57.828008 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac\": container with ID starting with bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac not found: ID does not exist" containerID="bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac" Jul 03 21:18:57.828078 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.828045 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac"} err="failed to get container status \"bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac\": rpc error: code = NotFound desc = could not find container \"bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac\": container with ID starting with bcd09f7a0c044602343c26f15c93c8fbae9d2d42db4eaa00dafab5b16ff504ac not found: ID does not exist" Jul 03 21:18:57.828078 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.828068 2558 scope.go:117] "RemoveContainer" containerID="ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf" Jul 03 21:18:57.828301 ip-10-0-131-199 kubenswrapper[2558]: E0703 21:18:57.828285 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf\": container with ID starting with ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf not found: ID does not exist" containerID="ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf" Jul 03 21:18:57.828355 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.828309 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf"} err="failed to get container status \"ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf\": rpc error: code = NotFound desc = could not find container \"ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf\": container with ID starting with ef566e48fbb69e8abf201be95c6bf8e030e1f58dd67499b1f2db834044fb01cf not found: ID does not exist" Jul 03 21:18:57.925468 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925440 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925468 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925464 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr9xn\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-kube-api-access-jr9xn\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925475 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-podinfo\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925484 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-envoy\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925492 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-credential-socket\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925501 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-workload-certs\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925509 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istiod-ca-cert\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925517 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-token\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:57.925615 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:57.925524 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc7640c-804e-4243-82ee-ed172ac5fbcc-istio-data\") on node \"ip-10-0-131-199.ec2.internal\" DevicePath \"\"" Jul 03 21:18:58.132014 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:58.131981 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr"] Jul 03 21:18:58.135827 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:58.135800 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1a4753-openshift-default-f8d97ffbc-fmpgr"] Jul 03 21:18:59.053026 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:18:59.052996 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" path="/var/lib/kubelet/pods/bdc7640c-804e-4243-82ee-ed172ac5fbcc/volumes" Jul 03 21:19:04.747832 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:04.747746 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-78bdbc9f8d-z54wh_d13720df-838d-46db-8c34-bfa952f4663b/maas-api/0.log" Jul 03 21:19:04.861271 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:04.861241 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718540-blnr9_b7655090-869f-44e0-ac6e-30c52e90e928/cleanup/0.log" Jul 03 21:19:04.972073 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:04.972046 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718555-cspwh_40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b/cleanup/0.log" Jul 03 21:19:05.085588 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:05.085559 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c7c44bb7-txjvx_eb784cff-9727-43cf-838a-af4b91f8a7bf/manager/0.log" Jul 03 21:19:05.434902 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:05.434822 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5448f4d7cf-kd78z_11e72f28-02b2-4452-9c99-0db7996119cb/manager/0.log" Jul 03 21:19:05.440128 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:05.440109 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5448f4d7cf-kd78z_11e72f28-02b2-4452-9c99-0db7996119cb/copy-manifests/0.log" Jul 03 21:19:05.823515 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:05.823489 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-npkv8_70990dd4-bc02-44f9-860e-67007986badb/postgres/0.log" Jul 03 21:19:06.580376 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.580318 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56_2b737fc0-5a2c-448c-95cf-1e86361c3610/extract/0.log" Jul 03 21:19:06.585601 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.585578 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56_2b737fc0-5a2c-448c-95cf-1e86361c3610/util/0.log" Jul 03 21:19:06.590969 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.590948 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56_2b737fc0-5a2c-448c-95cf-1e86361c3610/pull/0.log" Jul 03 21:19:06.697537 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.697517 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c_9d7c9fbf-1a21-4f60-840f-ab563bda1e98/pull/0.log" Jul 03 21:19:06.703159 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.703141 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c_9d7c9fbf-1a21-4f60-840f-ab563bda1e98/extract/0.log" Jul 03 21:19:06.708167 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.708149 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c_9d7c9fbf-1a21-4f60-840f-ab563bda1e98/util/0.log" Jul 03 21:19:06.818414 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.818387 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj_7c6c46d6-2abe-4693-ae94-608e27c0c2af/extract/0.log" Jul 03 21:19:06.823704 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.823685 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj_7c6c46d6-2abe-4693-ae94-608e27c0c2af/util/0.log" Jul 03 21:19:06.828555 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.828537 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj_7c6c46d6-2abe-4693-ae94-608e27c0c2af/pull/0.log" Jul 03 21:19:06.935847 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.935788 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm_47890d2f-69ac-4d41-9dd7-1f755a28414e/util/0.log" Jul 03 21:19:06.941201 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.941186 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm_47890d2f-69ac-4d41-9dd7-1f755a28414e/pull/0.log" Jul 03 21:19:06.946188 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:06.946170 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm_47890d2f-69ac-4d41-9dd7-1f755a28414e/extract/0.log" Jul 03 21:19:08.361811 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:08.361785 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qsftz_8cff6673-c65f-43b9-aa62-992749052eaf/discovery/0.log" Jul 03 21:19:08.829901 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:08.829871 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-x8pl2_e4283f37-329c-4daf-9528-636b9bb18729/payload-pre-processing/0.log" Jul 03 21:19:08.950236 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:08.950211 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nxcg8_85d86a36-e625-4b48-8261-e53315496ef5/payload-processing/0.log" Jul 03 21:19:09.622216 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:09.622184 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jkmjm_97fb9f2e-8a59-4cd1-a6d5-e9ca579128da/main/0.log" Jul 03 21:19:16.845443 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:16.845409 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fcjwp_fa3ffbd6-36ec-4ae6-8b13-09ffec020c70/global-pull-secret-syncer/0.log" Jul 03 21:19:16.988949 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:16.988916 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-84zjd_efc54219-f35d-4cbf-add6-4d42d71a6660/konnectivity-agent/0.log" Jul 03 21:19:17.079949 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:17.079921 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-199.ec2.internal_1474223590267aed250fed358f37a12c/haproxy/0.log" Jul 03 21:19:21.648671 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.648643 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56_2b737fc0-5a2c-448c-95cf-1e86361c3610/extract/0.log" Jul 03 21:19:21.664873 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.664849 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56_2b737fc0-5a2c-448c-95cf-1e86361c3610/util/0.log" Jul 03 21:19:21.680710 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.680691 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tkb56_2b737fc0-5a2c-448c-95cf-1e86361c3610/pull/0.log" Jul 03 21:19:21.701135 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.701114 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c_9d7c9fbf-1a21-4f60-840f-ab563bda1e98/extract/0.log" Jul 03 21:19:21.717738 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.717722 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c_9d7c9fbf-1a21-4f60-840f-ab563bda1e98/util/0.log" Jul 03 21:19:21.734101 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.734072 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0p9f9c_9d7c9fbf-1a21-4f60-840f-ab563bda1e98/pull/0.log" Jul 03 21:19:21.754122 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.754103 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj_7c6c46d6-2abe-4693-ae94-608e27c0c2af/extract/0.log" Jul 03 21:19:21.769471 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.769451 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj_7c6c46d6-2abe-4693-ae94-608e27c0c2af/util/0.log" Jul 03 21:19:21.784419 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.784398 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t2vdj_7c6c46d6-2abe-4693-ae94-608e27c0c2af/pull/0.log" Jul 03 21:19:21.804919 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.804899 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm_47890d2f-69ac-4d41-9dd7-1f755a28414e/extract/0.log" Jul 03 21:19:21.820186 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.820167 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm_47890d2f-69ac-4d41-9dd7-1f755a28414e/util/0.log" Jul 03 21:19:21.835193 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:21.835178 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1f4dfm_47890d2f-69ac-4d41-9dd7-1f755a28414e/pull/0.log" Jul 03 21:19:24.621592 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:24.621559 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/1.log" Jul 03 21:19:24.695640 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:24.695610 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-hx74w_c5e6301c-4b15-4210-aaf3-3bcf17c9c0fe/cluster-monitoring-operator/0.log" Jul 03 21:19:24.861884 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:24.861851 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26wsk_14ee3481-359b-443f-8dad-76c0a61e57c6/node-exporter/0.log" Jul 03 21:19:24.882634 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:24.882572 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26wsk_14ee3481-359b-443f-8dad-76c0a61e57c6/kube-rbac-proxy/0.log" Jul 03 21:19:24.899817 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:24.899800 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-26wsk_14ee3481-359b-443f-8dad-76c0a61e57c6/init-textfile/0.log" Jul 03 21:19:25.268843 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:25.268742 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-m5qjh_9d226421-b1ad-4b46-9145-eb9332d73f4a/prometheus-operator-admission-webhook/0.log" Jul 03 21:19:25.356700 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:25.356670 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85697c47cc-gdchw_f8a52ff3-7341-498e-a312-aa464d65aa34/thanos-query/0.log" Jul 03 21:19:25.372940 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:25.372913 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85697c47cc-gdchw_f8a52ff3-7341-498e-a312-aa464d65aa34/kube-rbac-proxy-web/0.log" Jul 03 21:19:25.388511 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:25.388474 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85697c47cc-gdchw_f8a52ff3-7341-498e-a312-aa464d65aa34/kube-rbac-proxy/0.log" Jul 03 21:19:25.404581 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:25.404557 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85697c47cc-gdchw_f8a52ff3-7341-498e-a312-aa464d65aa34/prom-label-proxy/0.log" Jul 03 21:19:25.420683 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:25.420664 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85697c47cc-gdchw_f8a52ff3-7341-498e-a312-aa464d65aa34/kube-rbac-proxy-rules/0.log" Jul 03 21:19:25.435911 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:25.435888 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-85697c47cc-gdchw_f8a52ff3-7341-498e-a312-aa464d65aa34/kube-rbac-proxy-metrics/0.log" Jul 03 21:19:26.555525 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:26.555502 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-xz8vm_377561ea-6240-45a1-820b-b0971f7f56a2/networking-console-plugin/0.log" Jul 03 21:19:27.329793 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.329750 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq"] Jul 03 21:19:27.330290 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330235 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a" containerName="cleanup" Jul 03 21:19:27.330290 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330258 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a" containerName="cleanup" Jul 03 21:19:27.330290 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330273 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" Jul 03 21:19:27.330290 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330282 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330300 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b" containerName="cleanup" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330309 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b" containerName="cleanup" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330326 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6078d95-e0be-41e9-a576-262023435ddc" containerName="maas-api" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330334 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6078d95-e0be-41e9-a576-262023435ddc" containerName="maas-api" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330348 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a10dd446-9bf9-4ba3-85f0-668ea6754de0" containerName="cleanup" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330357 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a10dd446-9bf9-4ba3-85f0-668ea6754de0" containerName="cleanup" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330451 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6078d95-e0be-41e9-a576-262023435ddc" containerName="maas-api" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330463 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e4bb2dc-545d-49a2-b58c-c6c0cd59a54a" containerName="cleanup" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330474 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b" containerName="cleanup" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330483 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330493 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc7640c-804e-4243-82ee-ed172ac5fbcc" containerName="istio-proxy" Jul 03 21:19:27.330565 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.330501 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a10dd446-9bf9-4ba3-85f0-668ea6754de0" containerName="cleanup" Jul 03 21:19:27.333605 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.333589 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.335989 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.335969 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nbmfn\"/\"kube-root-ca.crt\"" Jul 03 21:19:27.336919 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.336901 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nbmfn\"/\"default-dockercfg-sb5rk\"" Jul 03 21:19:27.336919 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.336908 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nbmfn\"/\"openshift-service-ca.crt\"" Jul 03 21:19:27.341935 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.341914 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq"] Jul 03 21:19:27.363373 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.363351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-podres\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.363471 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.363391 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lmxn\" (UniqueName: \"kubernetes.io/projected/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-kube-api-access-2lmxn\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.363471 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.363417 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-proc\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.363549 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.363494 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-lib-modules\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.363549 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.363517 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-sys\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.464802 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.464757 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-podres\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.464935 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.464826 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lmxn\" (UniqueName: \"kubernetes.io/projected/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-kube-api-access-2lmxn\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.464935 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.464868 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-proc\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.464935 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.464916 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-lib-modules\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.465105 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.464931 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-podres\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.465105 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.464952 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-sys\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.465105 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.464994 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-proc\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.465105 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.465014 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-sys\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.465105 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.465039 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-lib-modules\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.472643 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.472626 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lmxn\" (UniqueName: \"kubernetes.io/projected/6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc-kube-api-access-2lmxn\") pod \"perf-node-gather-daemonset-jpzdq\" (UID: \"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc\") " pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.627990 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.627919 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-sd2sl_417b55de-5169-4826-b16a-03a27dc86a73/download-server/0.log" Jul 03 21:19:27.644739 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.644721 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.770856 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.770824 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq"] Jul 03 21:19:27.772155 ip-10-0-131-199 kubenswrapper[2558]: W0703 21:19:27.772130 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6f3f88e2_4cc3_4f56_9cf7_95955e55a6cc.slice/crio-2db638f654462d809cc71d9de1100cf1f465b90481dcdc66a4f5fd319952e2ad WatchSource:0}: Error finding container 2db638f654462d809cc71d9de1100cf1f465b90481dcdc66a4f5fd319952e2ad: Status 404 returned error can't find the container with id 2db638f654462d809cc71d9de1100cf1f465b90481dcdc66a4f5fd319952e2ad Jul 03 21:19:27.975625 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.975536 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" event={"ID":"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc","Type":"ContainerStarted","Data":"916fde09bb275a755742164d5f08b5de811c973f697d6474a798656c9bb65513"} Jul 03 21:19:27.975625 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.975589 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" event={"ID":"6f3f88e2-4cc3-4f56-9cf7-95955e55a6cc","Type":"ContainerStarted","Data":"2db638f654462d809cc71d9de1100cf1f465b90481dcdc66a4f5fd319952e2ad"} Jul 03 21:19:27.975625 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.975612 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:27.988667 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:27.988620 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" podStartSLOduration=0.988605561 podStartE2EDuration="988.605561ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 21:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 21:19:27.988115782 +0000 UTC m=+2539.485683463" watchObservedRunningTime="2026-07-03 21:19:27.988605561 +0000 UTC m=+2539.486173221" Jul 03 21:19:29.220337 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:29.220309 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rpcz2_4798d9f3-76b8-471b-a6dc-e4882536cb4e/dns/0.log" Jul 03 21:19:29.235725 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:29.235705 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rpcz2_4798d9f3-76b8-471b-a6dc-e4882536cb4e/kube-rbac-proxy/0.log" Jul 03 21:19:29.272476 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:29.272458 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zb5jf_e99d1d0f-5a9b-4dea-af3d-463f3e45d615/dns-node-resolver/0.log" Jul 03 21:19:29.837473 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:29.837443 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nl4pk_d0583e0b-4471-4710-ba0f-5c59e4173743/node-ca/0.log" Jul 03 21:19:30.816436 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:30.816405 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qsftz_8cff6673-c65f-43b9-aa62-992749052eaf/discovery/0.log" Jul 03 21:19:31.027668 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:31.027639 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-x8pl2_e4283f37-329c-4daf-9528-636b9bb18729/payload-pre-processing/0.log" Jul 03 21:19:31.072809 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:31.072697 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nxcg8_85d86a36-e625-4b48-8261-e53315496ef5/payload-processing/0.log" Jul 03 21:19:31.744611 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:31.744575 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hqvwk_fa57c721-8f8e-4496-afec-72a573a690b4/serve-healthcheck-canary/0.log" Jul 03 21:19:32.191463 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:32.191435 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-5dbt7_0e0b6df9-93cb-4d46-b0a1-c7318a298782/insights-operator/0.log" Jul 03 21:19:32.191978 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:32.191926 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-5dbt7_0e0b6df9-93cb-4d46-b0a1-c7318a298782/insights-operator/1.log" Jul 03 21:19:32.260299 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:32.260275 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mqll_c7c1946a-264d-4401-b552-12914faa5d86/kube-rbac-proxy/0.log" Jul 03 21:19:32.275411 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:32.275388 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mqll_c7c1946a-264d-4401-b552-12914faa5d86/exporter/0.log" Jul 03 21:19:32.294447 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:32.294425 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mqll_c7c1946a-264d-4401-b552-12914faa5d86/extractor/0.log" Jul 03 21:19:33.991321 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:33.991296 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nbmfn/perf-node-gather-daemonset-jpzdq" Jul 03 21:19:34.656383 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:34.656354 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-78bdbc9f8d-z54wh_d13720df-838d-46db-8c34-bfa952f4663b/maas-api/0.log" Jul 03 21:19:34.671293 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:34.671263 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718540-blnr9_b7655090-869f-44e0-ac6e-30c52e90e928/cleanup/0.log" Jul 03 21:19:34.696204 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:34.696182 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29718555-cspwh_40b901b9-683a-4c8d-8bf5-6e1bb2b96f5b/cleanup/0.log" Jul 03 21:19:34.786771 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:34.786746 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c7c44bb7-txjvx_eb784cff-9727-43cf-838a-af4b91f8a7bf/manager/0.log" Jul 03 21:19:34.931140 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:34.931062 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5448f4d7cf-kd78z_11e72f28-02b2-4452-9c99-0db7996119cb/manager/0.log" Jul 03 21:19:34.947169 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:34.947147 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5448f4d7cf-kd78z_11e72f28-02b2-4452-9c99-0db7996119cb/copy-manifests/0.log" Jul 03 21:19:35.140361 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:35.140337 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-npkv8_70990dd4-bc02-44f9-860e-67007986badb/postgres/0.log" Jul 03 21:19:36.555336 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:36.555301 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66c99c87c9-knn78_dca714b8-d854-4852-a8d6-60383ac671cb/manager/0.log" Jul 03 21:19:37.763869 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:37.763783 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qsftz_8cff6673-c65f-43b9-aa62-992749052eaf/discovery/0.log" Jul 03 21:19:37.873775 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:37.873733 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-x8pl2_e4283f37-329c-4daf-9528-636b9bb18729/payload-pre-processing/0.log" Jul 03 21:19:37.906432 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:37.906408 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nxcg8_85d86a36-e625-4b48-8261-e53315496ef5/payload-processing/0.log" Jul 03 21:19:39.253252 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:39.253170 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qsftz_8cff6673-c65f-43b9-aa62-992749052eaf/discovery/0.log" Jul 03 21:19:39.383737 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:39.383711 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-x8pl2_e4283f37-329c-4daf-9528-636b9bb18729/payload-pre-processing/0.log" Jul 03 21:19:39.429961 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:39.429935 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-nxcg8_85d86a36-e625-4b48-8261-e53315496ef5/payload-processing/0.log" Jul 03 21:19:41.655010 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:41.654985 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sg576_00fe4a14-4e5c-4b68-bc91-9720552b5a1c/migrator/0.log" Jul 03 21:19:41.669737 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:41.669711 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-sg576_00fe4a14-4e5c-4b68-bc91-9720552b5a1c/graceful-termination/0.log" Jul 03 21:19:41.979914 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:41.979829 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-vdzf8_6638ddc3-acba-40eb-b82a-3ffeb3799103/kube-storage-version-migrator-operator/1.log" Jul 03 21:19:41.980627 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:41.980607 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-vdzf8_6638ddc3-acba-40eb-b82a-3ffeb3799103/kube-storage-version-migrator-operator/0.log" Jul 03 21:19:43.033782 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.033736 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kwnjh_7ea07ed9-94b3-45d7-a9b7-f0cb9955f423/kube-multus-additional-cni-plugins/0.log" Jul 03 21:19:43.049695 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.049672 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kwnjh_7ea07ed9-94b3-45d7-a9b7-f0cb9955f423/egress-router-binary-copy/0.log" Jul 03 21:19:43.065491 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.065467 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kwnjh_7ea07ed9-94b3-45d7-a9b7-f0cb9955f423/cni-plugins/0.log" Jul 03 21:19:43.083713 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.083694 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kwnjh_7ea07ed9-94b3-45d7-a9b7-f0cb9955f423/bond-cni-plugin/0.log" Jul 03 21:19:43.100259 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.100235 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kwnjh_7ea07ed9-94b3-45d7-a9b7-f0cb9955f423/routeoverride-cni/0.log" Jul 03 21:19:43.115867 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.115843 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kwnjh_7ea07ed9-94b3-45d7-a9b7-f0cb9955f423/whereabouts-cni-bincopy/0.log" Jul 03 21:19:43.132469 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.132451 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kwnjh_7ea07ed9-94b3-45d7-a9b7-f0cb9955f423/whereabouts-cni/0.log" Jul 03 21:19:43.287902 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.287883 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vv8r5_b74f1cc5-2656-4f91-b223-c5064e0611ff/kube-multus/0.log" Jul 03 21:19:43.334631 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.334605 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-85jm6_e3dec259-8bc3-4826-96aa-924c10061798/network-metrics-daemon/0.log" Jul 03 21:19:43.347603 ip-10-0-131-199 kubenswrapper[2558]: I0703 21:19:43.347582 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-85jm6_e3dec259-8bc3-4826-96aa-924c10061798/kube-rbac-proxy/0.log"