Sep 10 22:18:13.083078 ip-10-0-142-235 systemd[1]: Starting Kubernetes Kubelet... Sep 10 22:18:13.504010 ip-10-0-142-235 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 22:18:13.504010 ip-10-0-142-235 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 22:18:13.504010 ip-10-0-142-235 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 22:18:13.504010 ip-10-0-142-235 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 22:18:13.504010 ip-10-0-142-235 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 22:18:13.506193 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.506116 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 22:18:13.511371 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511350 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:18:13.511371 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511367 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:18:13.511371 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511372 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:18:13.511371 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511375 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:18:13.511371 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511378 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511381 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511384 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511387 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511389 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511392 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511395 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511397 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511400 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511403 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511405 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511408 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511410 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511413 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511416 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511418 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511421 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511423 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511426 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511428 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:18:13.511579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511431 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511434 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511437 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511441 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511447 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511449 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511452 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511456 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511460 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511463 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511466 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511469 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511471 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511475 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511478 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511481 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511484 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511487 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511490 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:18:13.512113 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511492 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511495 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511498 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511500 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511503 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511505 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511508 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511511 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511513 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511516 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511518 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511521 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511523 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511526 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511529 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511531 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511534 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511536 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511539 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511542 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:18:13.512579 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511544 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511547 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511549 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511551 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511554 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511557 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511559 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511563 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511565 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511568 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511570 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511573 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511575 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511578 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511581 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511584 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511586 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511589 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511592 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511594 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:18:13.513129 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511597 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511601 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511605 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511607 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511610 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511612 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511614 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511617 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511620 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511624 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511627 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511629 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511632 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511635 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511637 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.511640 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512346 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512352 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512355 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:18:13.513617 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512359 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512361 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512364 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512367 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512369 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512372 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512374 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512377 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512379 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512382 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512385 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512387 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512390 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512393 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512396 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512399 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512401 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512403 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512406 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:18:13.514093 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512408 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512411 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512414 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512417 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512419 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512422 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512425 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512427 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512430 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512433 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512435 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512438 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512440 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512443 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512445 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512447 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512450 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512452 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512455 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512458 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:18:13.514559 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512460 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512463 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512465 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512468 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512470 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512472 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512475 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512478 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512481 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512484 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512486 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512489 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512491 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512494 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512497 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512500 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512502 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512505 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512507 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512510 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:18:13.515069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512512 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512515 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512518 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512520 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512523 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512525 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512527 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512531 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512534 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512537 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512540 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512542 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512544 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512547 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512549 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512552 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512554 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512557 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512560 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512562 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:18:13.515581 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512565 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512569 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512571 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512574 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512576 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512579 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512582 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512585 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512590 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512594 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512596 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512599 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512602 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512605 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512608 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512611 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.512615 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513045 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:18:13.516079 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513052 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513055 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513058 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513062 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513065 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513068 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513070 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513073 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513076 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513079 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513082 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513085 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513088 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513091 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513093 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513097 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513101 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513105 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513108 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:18:13.516516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513111 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513113 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513117 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513119 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513122 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513124 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513127 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513129 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513132 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513135 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513138 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513140 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513143 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513146 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513149 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513152 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513155 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513157 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513160 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513162 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:18:13.517015 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513165 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513167 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513170 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513173 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513175 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513178 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513180 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513183 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513186 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513189 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513192 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513194 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513197 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513199 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513202 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513205 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513208 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513210 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513213 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513215 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:18:13.517591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513218 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513220 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513222 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513226 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513230 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513233 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513237 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513240 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513242 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513245 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513247 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513250 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513252 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513256 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513259 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513262 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513265 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513267 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513269 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:18:13.518132 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513272 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513275 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513278 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513281 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513283 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513286 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513289 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513291 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513294 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513296 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513299 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513301 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513303 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513306 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513308 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513311 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513313 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513315 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513318 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.513320 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:18:13.518577 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513387 2577 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513420 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513427 2577 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513431 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513435 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513438 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513442 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513446 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513449 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513452 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513457 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513461 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513466 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513476 2577 flags.go:64] FLAG: --cgroup-root="" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513481 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513484 2577 flags.go:64] FLAG: --client-ca-file="" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513488 2577 flags.go:64] FLAG: --cloud-provider="external" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513491 2577 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513495 2577 flags.go:64] FLAG: --cluster-domain="" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513498 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513501 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513504 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513507 2577 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513511 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 22:18:13.519082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513514 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513517 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513520 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513523 2577 flags.go:64] FLAG: --contention-profiling="false" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513526 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513528 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513531 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513534 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513538 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513541 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513544 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513548 2577 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513551 2577 flags.go:64] FLAG: --enable-server="true" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513554 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513558 2577 flags.go:64] FLAG: --event-burst="100" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513561 2577 flags.go:64] FLAG: --event-qps="50" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513564 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513567 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513569 2577 flags.go:64] FLAG: --eviction-hard="" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513573 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513576 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513580 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513583 2577 flags.go:64] FLAG: --eviction-soft="" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513586 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513588 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 22:18:13.519658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513592 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513594 2577 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513597 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513600 2577 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513603 2577 flags.go:64] FLAG: --feature-gates="" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513607 2577 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513609 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513612 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513615 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513618 2577 flags.go:64] FLAG: --healthz-port="10248" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513621 2577 flags.go:64] FLAG: --help="false" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513624 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513627 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513630 2577 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513633 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513636 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513639 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513642 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513647 2577 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513650 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513652 2577 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513655 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513658 2577 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513661 2577 flags.go:64] FLAG: --kube-reserved="" Sep 10 22:18:13.520295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513664 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513666 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513669 2577 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513672 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513675 2577 flags.go:64] FLAG: --lock-file="" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513678 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513681 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513684 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513689 2577 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513692 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513695 2577 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513698 2577 flags.go:64] FLAG: --logging-format="text" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513701 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513704 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513707 2577 flags.go:64] FLAG: --manifest-url="" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513710 2577 flags.go:64] FLAG: --manifest-url-header="" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513714 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513716 2577 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513721 2577 flags.go:64] FLAG: --max-pods="110" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513723 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513726 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513729 2577 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513732 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513735 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513737 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 22:18:13.520908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513740 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513751 2577 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513756 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513761 2577 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513765 2577 flags.go:64] FLAG: --pod-cidr="" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513769 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513774 2577 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513777 2577 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513780 2577 flags.go:64] FLAG: --pods-per-core="0" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513783 2577 flags.go:64] FLAG: --port="10250" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513786 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513788 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-017243119093439a4" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513794 2577 flags.go:64] FLAG: --qos-reserved="" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513797 2577 flags.go:64] FLAG: --read-only-port="10255" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513800 2577 flags.go:64] FLAG: --register-node="true" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513802 2577 flags.go:64] FLAG: --register-with-taints="" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513806 2577 flags.go:64] FLAG: --registry-burst="10" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513809 2577 flags.go:64] FLAG: --registry-qps="5" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513826 2577 flags.go:64] FLAG: --reserved-cpus="" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513829 2577 flags.go:64] FLAG: --reserved-memory="" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513837 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513841 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513843 2577 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513846 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513849 2577 flags.go:64] FLAG: --runonce="false" Sep 10 22:18:13.521502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513852 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513855 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513858 2577 flags.go:64] FLAG: --seccomp-default="false" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513861 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513864 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513867 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513869 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513872 2577 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513876 2577 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513879 2577 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513882 2577 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513885 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513888 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513891 2577 flags.go:64] FLAG: --system-cgroups="" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513893 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513898 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513901 2577 flags.go:64] FLAG: --tls-cert-file="" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513904 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513907 2577 flags.go:64] FLAG: --tls-min-version="" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513912 2577 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513915 2577 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513918 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513921 2577 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513924 2577 flags.go:64] FLAG: --v="2" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513928 2577 flags.go:64] FLAG: --version="false" Sep 10 22:18:13.522157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513932 2577 flags.go:64] FLAG: --vmodule="" Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513936 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.513939 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514051 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514055 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514058 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514061 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514064 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514068 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514071 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514074 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514076 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514079 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514082 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514084 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514088 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514091 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514093 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514096 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:18:13.522745 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514099 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514102 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514105 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514107 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514110 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514113 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514115 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514119 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514122 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514124 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514127 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514130 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514132 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514135 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514138 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514141 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514143 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514146 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514149 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514152 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:18:13.523231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514154 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514157 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514160 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514162 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514165 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514167 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514170 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514173 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514176 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514179 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514182 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514185 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514195 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514198 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514200 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514203 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514206 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514208 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514211 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514215 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:18:13.523723 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514217 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514220 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514222 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514225 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514228 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514230 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514233 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514236 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514240 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514244 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514247 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514249 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514253 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514257 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514260 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514263 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514265 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514268 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514270 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:18:13.524231 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514273 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514277 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514280 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514282 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514286 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514290 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514293 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514295 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514298 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514301 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514304 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514307 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514311 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514314 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514317 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514320 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514323 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514325 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514328 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514331 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:18:13.524734 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514335 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514338 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514341 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.514343 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.514943 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.521954 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.521970 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.521987 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.521993 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522046 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522050 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522053 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522056 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522059 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:18:13.525242 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522062 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522064 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522067 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522070 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522073 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522075 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522078 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522080 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522083 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522085 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522088 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522090 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522093 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522097 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522101 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522104 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522107 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522110 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522112 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:18:13.525592 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522115 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522118 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522121 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522137 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522141 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522145 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522149 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522152 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522155 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522158 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522161 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522163 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522166 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522169 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522172 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522174 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522177 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522179 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522182 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522184 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:18:13.526069 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522187 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522190 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522192 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522195 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522197 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522200 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522203 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522205 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522210 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522213 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522216 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522219 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522222 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522225 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522228 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522231 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522234 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522237 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522240 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522242 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:18:13.526545 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522245 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522248 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522251 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522253 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522256 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522258 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522261 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522263 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522265 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522268 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522270 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522273 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522276 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522278 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522281 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522283 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522286 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522290 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522293 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:18:13.527052 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522296 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522298 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522301 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522304 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522306 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522308 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522311 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522314 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522316 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522319 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522321 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522324 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522327 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522329 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522332 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:18:13.527502 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522334 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.522339 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522440 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522445 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522447 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522450 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522453 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522456 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522458 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522461 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522463 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522466 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522468 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522471 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522473 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 22:18:13.527955 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522476 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522478 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522481 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522483 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522486 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522489 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522491 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522494 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522496 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522499 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522502 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522504 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522507 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522509 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522512 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522514 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522516 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522519 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522521 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522524 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 22:18:13.528319 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522526 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522529 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522531 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522533 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522536 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522538 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522541 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522543 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522546 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522548 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522551 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522553 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522555 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522558 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522560 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522564 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522568 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522571 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522574 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522576 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 22:18:13.528804 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522579 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522581 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522584 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522587 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522589 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522592 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522594 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522597 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522599 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522602 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522604 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522607 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522609 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522612 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522616 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522619 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522621 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522624 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522627 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522629 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 22:18:13.529310 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522632 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522635 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522638 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522640 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522643 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522645 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522648 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522651 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522653 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522656 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522658 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522661 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522664 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522666 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522668 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522671 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522675 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522677 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522680 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 22:18:13.529784 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522683 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522686 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522688 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522690 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522693 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522695 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:13.522698 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.522702 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.522791 2577 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.524674 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.525865 2577 server.go:1019] "Starting client certificate rotation" Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.525957 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.525986 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 22:18:13.530249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.529262 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 22:18:13.530702 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.530331 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 22:18:13.541741 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.541719 2577 log.go:25] "Validated CRI v1 runtime API" Sep 10 22:18:13.544555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.544540 2577 log.go:25] "Validated CRI v1 image API" Sep 10 22:18:13.545450 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.545433 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 22:18:13.551870 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.551844 2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c4cc08ca-383c-443f-a320-103db1e0b382:/dev/nvme0n1p4 ccd9a906-e38c-4c87-aae6-88e24e00fe80:/dev/nvme0n1p3] Sep 10 22:18:13.551939 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.551870 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 22:18:13.557785 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.557680 2577 manager.go:216] Machine: {Timestamp:2026-09-10 22:18:13.556085876 +0000 UTC m=+0.352301348 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101962 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2180665768e18be2301382ae61df11 SystemUUID:ec218066-5768-e18b-e230-1382ae61df11 BootID:d99155ca-1238-48de-bae0-78d13315011a Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:11:ac:17:25:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:11:ac:17:25:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:bf:cc:29:3d:df Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 22:18:13.557785 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.557782 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 22:18:13.557939 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.557922 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 22:18:13.560084 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.560068 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 22:18:13.561516 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.561488 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 22:18:13.561684 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.561522 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 22:18:13.562716 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.562705 2577 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 22:18:13.562753 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.562719 2577 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 22:18:13.562753 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.562726 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 22:18:13.563455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.563444 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 22:18:13.564399 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.564389 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 22:18:13.567690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.567679 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 10 22:18:13.567805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.567796 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 22:18:13.567863 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.567840 2577 kubelet.go:486] "Attempting to sync node with API server" Sep 10 22:18:13.567904 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.567862 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 22:18:13.567904 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.567883 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 22:18:13.567904 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.567894 2577 kubelet.go:398] "Adding apiserver pod source" Sep 10 22:18:13.568043 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.567917 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 22:18:13.568275 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.568263 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 22:18:13.568309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.568282 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 22:18:13.570484 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.570468 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 22:18:13.572909 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.572895 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 22:18:13.572979 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.572943 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 22:18:13.575041 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575025 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 22:18:13.575041 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575042 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575049 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575054 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575063 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575068 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575081 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575086 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575092 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575097 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575114 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 22:18:13.575133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575119 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 22:18:13.575855 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575842 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 22:18:13.575855 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.575851 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 22:18:13.578605 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.578581 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 22:18:13.578890 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.578861 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 22:18:13.579090 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.579040 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 22:18:13.580374 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.580358 2577 server.go:1268] "Started kubelet" Sep 10 22:18:13.580461 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.580437 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 22:18:13.580641 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.580600 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 22:18:13.580745 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.580667 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 22:18:13.581159 ip-10-0-142-235 systemd[1]: Started Kubernetes Kubelet. Sep 10 22:18:13.581734 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.581542 2577 server.go:310] "Adding debug handlers to kubelet server" Sep 10 22:18:13.582513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.582495 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 22:18:13.587052 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.587031 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 22:18:13.587116 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.585887 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-235.ec2.internal.18d41597c61326e2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-235.ec2.internal,UID:ip-10-0-142-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-235.ec2.internal,},FirstTimestamp:2026-09-10 22:18:13.580318434 +0000 UTC m=+0.376533935,LastTimestamp:2026-09-10 22:18:13.580318434 +0000 UTC m=+0.376533935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-235.ec2.internal,}" Sep 10 22:18:13.587928 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.587915 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 22:18:13.587980 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.587927 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 22:18:13.588037 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.587997 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:13.588487 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.588474 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 22:18:13.588561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.588504 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 22:18:13.588614 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.588561 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 22:18:13.588673 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.588655 2577 reconstruct.go:97] "Volume reconstruction finished" Sep 10 22:18:13.588721 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.588674 2577 reconciler.go:29] "Reconciler: start to sync state" Sep 10 22:18:13.588928 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.588909 2577 factory.go:55] Registering systemd factory Sep 10 22:18:13.588928 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.588928 2577 factory.go:223] Registration of the systemd container factory successfully Sep 10 22:18:13.589201 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.589189 2577 factory.go:153] Registering CRI-O factory Sep 10 22:18:13.589245 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.589204 2577 factory.go:223] Registration of the crio container factory successfully Sep 10 22:18:13.589280 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.589248 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 22:18:13.589280 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.589264 2577 factory.go:103] Registering Raw factory Sep 10 22:18:13.589280 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.589274 2577 manager.go:1196] Started watching for new ooms in manager Sep 10 22:18:13.589603 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.589593 2577 manager.go:349] Starting recovery of all containers Sep 10 22:18:13.599491 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.599425 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 22:18:13.602249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.602230 2577 manager.go:354] Recovery completed Sep 10 22:18:13.602545 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.602524 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 22:18:13.606922 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.606909 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:18:13.608683 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.608653 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:18:13.608747 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.608694 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:18:13.608747 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.608704 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:18:13.609244 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.609231 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 22:18:13.609244 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.609238 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:18:13.609244 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.609243 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 22:18:13.609400 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.609261 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 10 22:18:13.609923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.609902 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6h5s5" Sep 10 22:18:13.610016 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.609982 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:18:13.610016 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.610010 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:18:13.610108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.610020 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:18:13.610580 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.610569 2577 policy_none.go:49] "None policy: Start" Sep 10 22:18:13.610616 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.610584 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 22:18:13.610616 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.610594 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 22:18:13.611092 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.611022 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-235.ec2.internal.18d41597c7c3ef23 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-235.ec2.internal,UID:ip-10-0-142-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-235.ec2.internal,},FirstTimestamp:2026-09-10 22:18:13.608681251 +0000 UTC m=+0.404896719,LastTimestamp:2026-09-10 22:18:13.608681251 +0000 UTC m=+0.404896719,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-235.ec2.internal,}" Sep 10 22:18:13.611912 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.611902 2577 policy_none.go:47] "Start" Sep 10 22:18:13.618074 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.618057 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6h5s5" Sep 10 22:18:13.618760 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.618677 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-235.ec2.internal.18d41597c7c43716 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-235.ec2.internal,UID:ip-10-0-142-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-235.ec2.internal,},FirstTimestamp:2026-09-10 22:18:13.60869967 +0000 UTC m=+0.404915138,LastTimestamp:2026-09-10 22:18:13.60869967 +0000 UTC m=+0.404915138,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-235.ec2.internal,}" Sep 10 22:18:13.621037 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.620981 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-235.ec2.internal.18d41597c7c457fe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-235.ec2.internal,UID:ip-10-0-142-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-235.ec2.internal,},FirstTimestamp:2026-09-10 22:18:13.608708094 +0000 UTC m=+0.404923562,LastTimestamp:2026-09-10 22:18:13.608708094 +0000 UTC m=+0.404923562,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-235.ec2.internal,}" Sep 10 22:18:13.651497 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.651379 2577 manager.go:337] "Starting Device Plugin manager" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.651522 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.651536 2577 server.go:87] "Starting device plugin registration server" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.651767 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.651776 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.651896 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.651995 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.652006 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.652552 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 22:18:13.667145 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.652604 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:13.741169 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.741143 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 22:18:13.742514 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.742493 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 22:18:13.742590 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.742544 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 22:18:13.742590 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.742569 2577 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 22:18:13.742663 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.742649 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 22:18:13.744998 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.744982 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:18:13.751866 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.751847 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:18:13.752695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.752670 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:18:13.752788 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.752705 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:18:13.752788 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.752725 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:18:13.752788 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.752744 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.761692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.761621 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.761692 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.761644 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-235.ec2.internal\": node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:13.787048 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.787023 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:13.843724 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.843698 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal"] Sep 10 22:18:13.843795 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.843772 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:18:13.845615 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.845594 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:18:13.845693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.845624 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:18:13.845693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.845633 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:18:13.846949 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.846936 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:18:13.847064 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847051 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.847121 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847076 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:18:13.847748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847733 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:18:13.847807 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847762 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:18:13.847807 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847771 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:18:13.847894 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847733 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:18:13.847894 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847856 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:18:13.847894 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.847883 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:18:13.848938 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.848923 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.849015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.848952 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 22:18:13.849653 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.849641 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientMemory" Sep 10 22:18:13.849723 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.849660 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 22:18:13.849723 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.849668 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeHasSufficientPID" Sep 10 22:18:13.875324 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.875308 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-235.ec2.internal\" not found" node="ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.879775 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.879759 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-235.ec2.internal\" not found" node="ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.887648 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.887633 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:13.890912 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.890897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e300852a2fc99e2c8b674aec6bd8a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal\" (UID: \"9e300852a2fc99e2c8b674aec6bd8a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.890977 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.890951 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be714b5e0febb4494d9a83009ae5a158-config\") pod \"kube-apiserver-proxy-ip-10-0-142-235.ec2.internal\" (UID: \"be714b5e0febb4494d9a83009ae5a158\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.890977 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.890972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9e300852a2fc99e2c8b674aec6bd8a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal\" (UID: \"9e300852a2fc99e2c8b674aec6bd8a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.988137 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:13.988107 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:13.991398 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.991381 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9e300852a2fc99e2c8b674aec6bd8a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal\" (UID: \"9e300852a2fc99e2c8b674aec6bd8a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.991461 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.991406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e300852a2fc99e2c8b674aec6bd8a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal\" (UID: \"9e300852a2fc99e2c8b674aec6bd8a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.991461 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.991456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be714b5e0febb4494d9a83009ae5a158-config\") pod \"kube-apiserver-proxy-ip-10-0-142-235.ec2.internal\" (UID: \"be714b5e0febb4494d9a83009ae5a158\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.991558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.991480 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be714b5e0febb4494d9a83009ae5a158-config\") pod \"kube-apiserver-proxy-ip-10-0-142-235.ec2.internal\" (UID: \"be714b5e0febb4494d9a83009ae5a158\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.991558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.991486 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9e300852a2fc99e2c8b674aec6bd8a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal\" (UID: \"9e300852a2fc99e2c8b674aec6bd8a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:13.991558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:13.991482 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e300852a2fc99e2c8b674aec6bd8a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal\" (UID: \"9e300852a2fc99e2c8b674aec6bd8a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:14.088863 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.088836 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:14.178153 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.178126 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:14.182966 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.182948 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" Sep 10 22:18:14.189806 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.189788 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:14.290205 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.290181 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:14.390733 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.390673 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-235.ec2.internal\" not found" Sep 10 22:18:14.468970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.468948 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:18:14.489063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.489044 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" Sep 10 22:18:14.499054 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.499035 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 22:18:14.500727 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.500715 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" Sep 10 22:18:14.512850 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.512836 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 22:18:14.524914 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.524891 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 22:18:14.525023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.525003 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 22:18:14.525066 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.525036 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 22:18:14.568192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.568177 2577 apiserver.go:52] "Watching apiserver" Sep 10 22:18:14.585922 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.585900 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 22:18:14.586281 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.586259 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-g7tpk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x","openshift-cluster-node-tuning-operator/tuned-f595d","openshift-dns/node-resolver-nj85g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal","openshift-multus/multus-additional-cni-plugins-j454d","openshift-network-operator/iptables-alerter-pbf7c","openshift-ovn-kubernetes/ovnkube-node-slc2q","kube-system/konnectivity-agent-scrkj","kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal","openshift-image-registry/node-ca-rstb7","openshift-multus/multus-fcrdb","openshift-multus/network-metrics-daemon-9dbrx","openshift-network-diagnostics/network-check-target-cmpvq"] Sep 10 22:18:14.587858 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.587839 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.588054 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.588032 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 22:18:14.589068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.589049 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.589856 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.589839 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s8xk7\"" Sep 10 22:18:14.590000 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.589982 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 22:18:14.590044 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.589982 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:18:14.590128 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.590114 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 22:18:14.590272 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.590258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.590577 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.590564 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 22:18:14.590869 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.590854 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 22:18:14.590910 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.590867 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 22:18:14.590943 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.590923 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c8wjg\"" Sep 10 22:18:14.591696 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.591682 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.591765 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.591727 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 22:18:14.592031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.592018 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:18:14.592130 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.592116 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lplpg\"" Sep 10 22:18:14.593047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.593030 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:14.593121 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.593091 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:14.593452 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.593433 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 22:18:14.593502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.593486 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-858hx\"" Sep 10 22:18:14.593841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.593810 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 22:18:14.594399 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.594384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/147108fb-2bd0-4022-9d0c-1fae37ad557e-iptables-alerter-script\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.594506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.594411 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/147108fb-2bd0-4022-9d0c-1fae37ad557e-host-slash\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.594506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.594437 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57467133-81f3-434e-ae9f-1f07cd84ceba-host\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.594506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.594464 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.594657 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.594462 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px7jc\" (UniqueName: \"kubernetes.io/projected/147108fb-2bd0-4022-9d0c-1fae37ad557e-kube-api-access-px7jc\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.594657 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.594609 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57467133-81f3-434e-ae9f-1f07cd84ceba-serviceca\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.594657 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.594635 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glfqz\" (UniqueName: \"kubernetes.io/projected/57467133-81f3-434e-ae9f-1f07cd84ceba-kube-api-access-glfqz\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.596151 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596134 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:14.596198 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596180 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 22:18:14.596490 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596472 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 22:18:14.596564 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596504 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 22:18:14.596564 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596524 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 22:18:14.596564 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596532 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 22:18:14.596696 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596565 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-57pdx\"" Sep 10 22:18:14.596959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.596944 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 22:18:14.597553 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.597537 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:14.597624 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.597601 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:14.597897 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.597883 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 22:18:14.598026 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.598011 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rj4cd\"" Sep 10 22:18:14.598197 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.598181 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 22:18:14.599223 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.599208 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.600795 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.600778 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.602351 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.602329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.602446 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.602380 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:14.602446 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.602436 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 22:18:14.602446 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.602443 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 22:18:14.602611 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.602463 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 22:18:14.602907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.602891 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5b4mw\"" Sep 10 22:18:14.603457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603246 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 22:18:14.603457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603247 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5pksw\"" Sep 10 22:18:14.603457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603349 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 22:18:14.603626 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603611 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 22:18:14.603673 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.603731 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603721 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 22:18:14.603801 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603785 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 22:18:14.603887 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.603810 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 22:18:14.605580 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.605549 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 22:18:14.607094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.607075 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l2nxd\"" Sep 10 22:18:14.621717 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.621674 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 22:13:13 +0000 UTC" deadline="2028-04-18 09:17:06.51600535 +0000 UTC" Sep 10 22:18:14.621717 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.621720 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14050h58m51.894289608s" Sep 10 22:18:14.625575 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.625560 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-29ttz" Sep 10 22:18:14.633692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.633678 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-29ttz" Sep 10 22:18:14.689943 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.689928 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 22:18:14.694744 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57467133-81f3-434e-ae9f-1f07cd84ceba-host\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.694852 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-systemd-units\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.694852 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-etc-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.694852 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694782 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovnkube-script-lib\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.694852 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694798 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-cni-bin\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.694852 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-hostroot\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-systemd\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.695019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57467133-81f3-434e-ae9f-1f07cd84ceba-host\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.695019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694896 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovnkube-config\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.695019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.695019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:14.695019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.694989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-system-cni-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-run\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695043 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-lib-modules\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df8sl\" (UniqueName: \"kubernetes.io/projected/c356678a-1be3-4972-901f-d1b445f890d1-kube-api-access-df8sl\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695089 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl9gt\" (UniqueName: \"kubernetes.io/projected/a936c932-4a9f-4df8-8a50-3cd336664bff-kube-api-access-jl9gt\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-log-socket\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695148 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-cni-bin\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-cni-netd\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695180 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf48x\" (UniqueName: \"kubernetes.io/projected/5308bd19-fb64-46c0-95aa-db0504e62f76-kube-api-access-sf48x\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysconfig\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysctl-d\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpv2p\" (UniqueName: \"kubernetes.io/projected/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-kube-api-access-kpv2p\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0d3d831-2afd-4647-9c64-22c55eccf863-kubelet-config\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-cnibin\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-cni-multus\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-host\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c356678a-1be3-4972-901f-d1b445f890d1-etc-tuned\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-netns\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695394 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-etc-kubernetes\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-hosts-file\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57467133-81f3-434e-ae9f-1f07cd84ceba-serviceca\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-etc-selinux\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695503 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-run-netns\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695531 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-cnibin\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5308bd19-fb64-46c0-95aa-db0504e62f76-cni-binary-copy\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.695594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695600 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695614 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-env-overrides\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695629 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovn-node-metrics-cert\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0d3d831-2afd-4647-9c64-22c55eccf863-dbus\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695687 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-cni-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695714 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-kubernetes\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/147108fb-2bd0-4022-9d0c-1fae37ad557e-host-slash\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-slash\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-var-lib-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-cni-binary-copy\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695805 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/147108fb-2bd0-4022-9d0c-1fae37ad557e-host-slash\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-kubelet\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-systemd\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c356678a-1be3-4972-901f-d1b445f890d1-tmp\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695906 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px7jc\" (UniqueName: \"kubernetes.io/projected/147108fb-2bd0-4022-9d0c-1fae37ad557e-kube-api-access-px7jc\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.696287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695926 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-sys-fs\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-ovn\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695954 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.695979 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d2b6e10a-8490-4ef2-9146-601fb419ad66-konnectivity-ca\") pod \"konnectivity-agent-scrkj\" (UID: \"d2b6e10a-8490-4ef2-9146-601fb419ad66\") " pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-socket-dir-parent\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-sys\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696053 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-system-cni-dir\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d2b6e10a-8490-4ef2-9146-601fb419ad66-agent-certs\") pod \"konnectivity-agent-scrkj\" (UID: \"d2b6e10a-8490-4ef2-9146-601fb419ad66\") " pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2gzm\" (UniqueName: \"kubernetes.io/projected/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-kube-api-access-d2gzm\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696153 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-daemon-config\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysctl-conf\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696198 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-node-log\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696243 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6b27\" (UniqueName: \"kubernetes.io/projected/079e30d5-078c-4982-981d-f4df3a17834b-kube-api-access-w6b27\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696269 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-k8s-cni-cncf-io\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.696805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696290 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-modprobe-d\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696308 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-tmp-dir\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9bhs\" (UniqueName: \"kubernetes.io/projected/80749e7c-a14b-4f64-ac75-c35eb2f5b359-kube-api-access-b9bhs\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-conf-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-registration-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696389 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-device-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-kubelet\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696419 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696454 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-run-ovn-kubernetes\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-var-lib-kubelet\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/147108fb-2bd0-4022-9d0c-1fae37ad557e-iptables-alerter-script\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696518 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glfqz\" (UniqueName: \"kubernetes.io/projected/57467133-81f3-434e-ae9f-1f07cd84ceba-kube-api-access-glfqz\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-socket-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.696548 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-multus-certs\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.697336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.697041 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/147108fb-2bd0-4022-9d0c-1fae37ad557e-iptables-alerter-script\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.697862 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.697845 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57467133-81f3-434e-ae9f-1f07cd84ceba-serviceca\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.701556 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.701538 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 22:18:14.704496 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.704478 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px7jc\" (UniqueName: \"kubernetes.io/projected/147108fb-2bd0-4022-9d0c-1fae37ad557e-kube-api-access-px7jc\") pod \"iptables-alerter-pbf7c\" (UID: \"147108fb-2bd0-4022-9d0c-1fae37ad557e\") " pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.704558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.704479 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glfqz\" (UniqueName: \"kubernetes.io/projected/57467133-81f3-434e-ae9f-1f07cd84ceba-kube-api-access-glfqz\") pod \"node-ca-rstb7\" (UID: \"57467133-81f3-434e-ae9f-1f07cd84ceba\") " pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.797315 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797290 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysctl-d\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.797491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797324 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpv2p\" (UniqueName: \"kubernetes.io/projected/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-kube-api-access-kpv2p\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.797491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797345 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0d3d831-2afd-4647-9c64-22c55eccf863-kubelet-config\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.797491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797368 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-cnibin\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-cnibin\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797438 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d0d3d831-2afd-4647-9c64-22c55eccf863-kubelet-config\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.797491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797438 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysctl-d\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.797491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-cni-multus\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-host\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c356678a-1be3-4972-901f-d1b445f890d1-etc-tuned\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797548 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-cni-multus\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-netns\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-etc-kubernetes\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797601 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-netns\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797563 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-host\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-hosts-file\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797630 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-etc-kubernetes\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797643 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-etc-selinux\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797657 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-hosts-file\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797667 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-run-netns\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797688 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-etc-selinux\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797695 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-run-netns\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-cnibin\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.797844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5308bd19-fb64-46c0-95aa-db0504e62f76-cni-binary-copy\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-env-overrides\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797796 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-cnibin\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797854 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovn-node-metrics-cert\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0d3d831-2afd-4647-9c64-22c55eccf863-dbus\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-cni-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797987 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d0d3d831-2afd-4647-9c64-22c55eccf863-dbus\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.797998 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-cni-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-kubernetes\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-slash\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-var-lib-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-cni-binary-copy\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798125 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-slash\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-kubelet\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.798671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-systemd\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c356678a-1be3-4972-901f-d1b445f890d1-tmp\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798251 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-sys-fs\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-var-lib-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798275 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-ovn\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798225 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-kubernetes\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-env-overrides\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798331 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-systemd\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798349 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-sys-fs\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798362 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d2b6e10a-8490-4ef2-9146-601fb419ad66-konnectivity-ca\") pod \"konnectivity-agent-scrkj\" (UID: \"d2b6e10a-8490-4ef2-9146-601fb419ad66\") " pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798383 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-ovn\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-socket-dir-parent\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-sys\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798442 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-system-cni-dir\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d2b6e10a-8490-4ef2-9146-601fb419ad66-agent-certs\") pod \"konnectivity-agent-scrkj\" (UID: \"d2b6e10a-8490-4ef2-9146-601fb419ad66\") " pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798490 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:14.799602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gzm\" (UniqueName: \"kubernetes.io/projected/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-kube-api-access-d2gzm\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798541 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5308bd19-fb64-46c0-95aa-db0504e62f76-cni-binary-copy\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798598 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-socket-dir-parent\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798642 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-sys\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798677 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-system-cni-dir\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798769 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-cni-binary-copy\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-daemon-config\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798846 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysctl-conf\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798894 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-node-log\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6b27\" (UniqueName: \"kubernetes.io/projected/079e30d5-078c-4982-981d-f4df3a17834b-kube-api-access-w6b27\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.798976 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-k8s-cni-cncf-io\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-modprobe-d\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-tmp-dir\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799036 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-daemon-config\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9bhs\" (UniqueName: \"kubernetes.io/projected/80749e7c-a14b-4f64-ac75-c35eb2f5b359-kube-api-access-b9bhs\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-conf-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.800513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-registration-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799104 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d2b6e10a-8490-4ef2-9146-601fb419ad66-konnectivity-ca\") pod \"konnectivity-agent-scrkj\" (UID: \"d2b6e10a-8490-4ef2-9146-601fb419ad66\") " pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-device-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-kubelet\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799164 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799188 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-run-ovn-kubernetes\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-var-lib-kubelet\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799222 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-modprobe-d\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-socket-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-node-log\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-multus-certs\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-systemd-units\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-etc-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799367 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-kubelet\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovnkube-script-lib\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-cni-bin\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799416 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/079e30d5-078c-4982-981d-f4df3a17834b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.801562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799438 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-hostroot\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-systemd\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799465 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-k8s-cni-cncf-io\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovnkube-config\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.799552 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-system-cni-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-run\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.799628 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs podName:0521c5d3-62eb-4ea7-b2e4-a2da4244d291 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:15.29960635 +0000 UTC m=+2.095821830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs") pod "network-metrics-daemon-9dbrx" (UID: "0521c5d3-62eb-4ea7-b2e4-a2da4244d291") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799620 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-run-multus-certs\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799632 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-lib-modules\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799673 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df8sl\" (UniqueName: \"kubernetes.io/projected/c356678a-1be3-4972-901f-d1b445f890d1-kube-api-access-df8sl\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl9gt\" (UniqueName: \"kubernetes.io/projected/a936c932-4a9f-4df8-8a50-3cd336664bff-kube-api-access-jl9gt\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799718 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-lib-modules\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-log-socket\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799749 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.802433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-cni-bin\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799787 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-cni-netd\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799762 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-log-socket\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799896 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-tmp-dir\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799940 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-run\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799955 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.799999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf48x\" (UniqueName: \"kubernetes.io/projected/5308bd19-fb64-46c0-95aa-db0504e62f76-kube-api-access-sf48x\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-var-lib-kubelet\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800027 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysconfig\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800109 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysconfig\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800134 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/079e30d5-078c-4982-981d-f4df3a17834b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800162 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-kubelet\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-run-ovn-kubernetes\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800221 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-socket-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-cni-netd\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800236 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-device-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800270 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-host-var-lib-cni-bin\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-systemd-units\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800319 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c356678a-1be3-4972-901f-d1b445f890d1-etc-sysctl-conf\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800333 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-etc-openvswitch\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800352 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a936c932-4a9f-4df8-8a50-3cd336664bff-registration-dir\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800372 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-hostroot\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-multus-conf-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800448 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-host-cni-bin\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.800507 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800495 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/80749e7c-a14b-4f64-ac75-c35eb2f5b359-run-systemd\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.800567 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret podName:d0d3d831-2afd-4647-9c64-22c55eccf863 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:15.300551046 +0000 UTC m=+2.096766500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret") pod "global-pull-secret-syncer-g7tpk" (UID: "d0d3d831-2afd-4647-9c64-22c55eccf863") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800616 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5308bd19-fb64-46c0-95aa-db0504e62f76-system-cni-dir\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800811 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovnkube-script-lib\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800834 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c356678a-1be3-4972-901f-d1b445f890d1-etc-tuned\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800873 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c356678a-1be3-4972-901f-d1b445f890d1-tmp\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.800924 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovnkube-config\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.802596 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d2b6e10a-8490-4ef2-9146-601fb419ad66-agent-certs\") pod \"konnectivity-agent-scrkj\" (UID: \"d2b6e10a-8490-4ef2-9146-601fb419ad66\") " pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:14.803732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.803308 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/80749e7c-a14b-4f64-ac75-c35eb2f5b359-ovn-node-metrics-cert\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.812172 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.812152 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpv2p\" (UniqueName: \"kubernetes.io/projected/63da62fc-f1bc-49e9-b2ff-0a39cf81331f-kube-api-access-kpv2p\") pod \"node-resolver-nj85g\" (UID: \"63da62fc-f1bc-49e9-b2ff-0a39cf81331f\") " pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.814521 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.814499 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:18:14.814611 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.814526 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:18:14.814611 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.814539 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jrxkn for pod openshift-network-diagnostics/network-check-target-cmpvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:14.814722 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:14.814614 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn podName:6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d nodeName:}" failed. No retries permitted until 2026-09-10 22:18:15.314595867 +0000 UTC m=+2.110811335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jrxkn" (UniqueName: "kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn") pod "network-check-target-cmpvq" (UID: "6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:14.816957 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.816633 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl9gt\" (UniqueName: \"kubernetes.io/projected/a936c932-4a9f-4df8-8a50-3cd336664bff-kube-api-access-jl9gt\") pod \"aws-ebs-csi-driver-node-hz54x\" (UID: \"a936c932-4a9f-4df8-8a50-3cd336664bff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:14.817040 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.816977 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6b27\" (UniqueName: \"kubernetes.io/projected/079e30d5-078c-4982-981d-f4df3a17834b-kube-api-access-w6b27\") pod \"multus-additional-cni-plugins-j454d\" (UID: \"079e30d5-078c-4982-981d-f4df3a17834b\") " pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:14.817122 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.817104 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df8sl\" (UniqueName: \"kubernetes.io/projected/c356678a-1be3-4972-901f-d1b445f890d1-kube-api-access-df8sl\") pod \"tuned-f595d\" (UID: \"c356678a-1be3-4972-901f-d1b445f890d1\") " pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.817224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.817205 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf48x\" (UniqueName: \"kubernetes.io/projected/5308bd19-fb64-46c0-95aa-db0504e62f76-kube-api-access-sf48x\") pod \"multus-fcrdb\" (UID: \"5308bd19-fb64-46c0-95aa-db0504e62f76\") " pod="openshift-multus/multus-fcrdb" Sep 10 22:18:14.817269 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.817254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2gzm\" (UniqueName: \"kubernetes.io/projected/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-kube-api-access-d2gzm\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:14.817473 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.817457 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9bhs\" (UniqueName: \"kubernetes.io/projected/80749e7c-a14b-4f64-ac75-c35eb2f5b359-kube-api-access-b9bhs\") pod \"ovnkube-node-slc2q\" (UID: \"80749e7c-a14b-4f64-ac75-c35eb2f5b359\") " pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:14.876637 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:14.875894 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe714b5e0febb4494d9a83009ae5a158.slice/crio-40cb9a062f7aebefacd8c908cd78ad4022ffe1932e7fb2d1168203198a944f3b WatchSource:0}: Error finding container 40cb9a062f7aebefacd8c908cd78ad4022ffe1932e7fb2d1168203198a944f3b: Status 404 returned error can't find the container with id 40cb9a062f7aebefacd8c908cd78ad4022ffe1932e7fb2d1168203198a944f3b Sep 10 22:18:14.876752 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:14.876692 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e300852a2fc99e2c8b674aec6bd8a96.slice/crio-38daaa33b24b4c6841d2746fe560162e666d3d98abfdf753bd6829e16d722960 WatchSource:0}: Error finding container 38daaa33b24b4c6841d2746fe560162e666d3d98abfdf753bd6829e16d722960: Status 404 returned error can't find the container with id 38daaa33b24b4c6841d2746fe560162e666d3d98abfdf753bd6829e16d722960 Sep 10 22:18:14.881291 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.881275 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:18:14.893617 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.893593 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:18:14.921982 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.921943 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pbf7c" Sep 10 22:18:14.927544 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.927530 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:18:14.928953 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:14.928934 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147108fb_2bd0_4022_9d0c_1fae37ad557e.slice/crio-c8fb6541911663d97932ecdfd55e349dec2407e77f2b909af6adbfd0603dd8ca WatchSource:0}: Error finding container c8fb6541911663d97932ecdfd55e349dec2407e77f2b909af6adbfd0603dd8ca: Status 404 returned error can't find the container with id c8fb6541911663d97932ecdfd55e349dec2407e77f2b909af6adbfd0603dd8ca Sep 10 22:18:14.939114 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.939100 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rstb7" Sep 10 22:18:14.945878 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:14.945860 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57467133_81f3_434e_ae9f_1f07cd84ceba.slice/crio-ef7904b85787da024e97e448707d0c83f63f91af4946b7b7fae422f73b857fc8 WatchSource:0}: Error finding container ef7904b85787da024e97e448707d0c83f63f91af4946b7b7fae422f73b857fc8: Status 404 returned error can't find the container with id ef7904b85787da024e97e448707d0c83f63f91af4946b7b7fae422f73b857fc8 Sep 10 22:18:14.955951 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.955936 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f595d" Sep 10 22:18:14.961976 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:14.961956 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc356678a_1be3_4972_901f_d1b445f890d1.slice/crio-831ce1587b866e6acb0af52004ad07dc796cf0ab70b06ed7bb18d256d1070cd2 WatchSource:0}: Error finding container 831ce1587b866e6acb0af52004ad07dc796cf0ab70b06ed7bb18d256d1070cd2: Status 404 returned error can't find the container with id 831ce1587b866e6acb0af52004ad07dc796cf0ab70b06ed7bb18d256d1070cd2 Sep 10 22:18:14.974703 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.974688 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nj85g" Sep 10 22:18:14.981477 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:14.981460 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63da62fc_f1bc_49e9_b2ff_0a39cf81331f.slice/crio-b0fd46c81dd4e76a110c98745c966faa155627cc3ae79b8802f9ccaaf3486ca3 WatchSource:0}: Error finding container b0fd46c81dd4e76a110c98745c966faa155627cc3ae79b8802f9ccaaf3486ca3: Status 404 returned error can't find the container with id b0fd46c81dd4e76a110c98745c966faa155627cc3ae79b8802f9ccaaf3486ca3 Sep 10 22:18:14.997560 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:14.997544 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j454d" Sep 10 22:18:15.003862 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:15.003846 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod079e30d5_078c_4982_981d_f4df3a17834b.slice/crio-4105357b3db92d3c47f32087c582f36e34b4e92bb2a104025e128cab9756b90d WatchSource:0}: Error finding container 4105357b3db92d3c47f32087c582f36e34b4e92bb2a104025e128cab9756b90d: Status 404 returned error can't find the container with id 4105357b3db92d3c47f32087c582f36e34b4e92bb2a104025e128cab9756b90d Sep 10 22:18:15.012611 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.012595 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:15.018384 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:15.018369 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b6e10a_8490_4ef2_9146_601fb419ad66.slice/crio-6f417d77298c073e9a83c96517395743f541613a5875c7dd15d86b9dd426717c WatchSource:0}: Error finding container 6f417d77298c073e9a83c96517395743f541613a5875c7dd15d86b9dd426717c: Status 404 returned error can't find the container with id 6f417d77298c073e9a83c96517395743f541613a5875c7dd15d86b9dd426717c Sep 10 22:18:15.030546 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.030528 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" Sep 10 22:18:15.038162 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.038146 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:15.038718 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:15.038704 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda936c932_4a9f_4df8_8a50_3cd336664bff.slice/crio-d07f7042b5a9335b28ffb9c83c9e8c1d44b0dedcef37eddc0b9b9c257c1b5882 WatchSource:0}: Error finding container d07f7042b5a9335b28ffb9c83c9e8c1d44b0dedcef37eddc0b9b9c257c1b5882: Status 404 returned error can't find the container with id d07f7042b5a9335b28ffb9c83c9e8c1d44b0dedcef37eddc0b9b9c257c1b5882 Sep 10 22:18:15.040513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.040495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fcrdb" Sep 10 22:18:15.044776 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:15.044760 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80749e7c_a14b_4f64_ac75_c35eb2f5b359.slice/crio-951430e9770798b164e99ab3bd2c62ff65bed732fcdeed7e27ddac6e4693fc21 WatchSource:0}: Error finding container 951430e9770798b164e99ab3bd2c62ff65bed732fcdeed7e27ddac6e4693fc21: Status 404 returned error can't find the container with id 951430e9770798b164e99ab3bd2c62ff65bed732fcdeed7e27ddac6e4693fc21 Sep 10 22:18:15.047387 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:15.047368 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5308bd19_fb64_46c0_95aa_db0504e62f76.slice/crio-5ae1258c21828894110b72d68701166c817e3eb1137da5d3040f23e0d5dc39dd WatchSource:0}: Error finding container 5ae1258c21828894110b72d68701166c817e3eb1137da5d3040f23e0d5dc39dd: Status 404 returned error can't find the container with id 5ae1258c21828894110b72d68701166c817e3eb1137da5d3040f23e0d5dc39dd Sep 10 22:18:15.048752 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.048725 2577 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-multus start failed in pod multus-fcrdb_openshift-multus(5308bd19-fb64-46c0-95aa-db0504e62f76): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Sep 10 22:18:15.048848 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.048760 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-fcrdb" podUID="5308bd19-fb64-46c0-95aa-db0504e62f76" Sep 10 22:18:15.304409 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.304384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:15.304538 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.304423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:15.304538 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.304506 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:15.304538 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.304513 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:15.304626 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.304551 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret podName:d0d3d831-2afd-4647-9c64-22c55eccf863 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:16.304535273 +0000 UTC m=+3.100750739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret") pod "global-pull-secret-syncer-g7tpk" (UID: "d0d3d831-2afd-4647-9c64-22c55eccf863") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:15.304626 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.304563 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs podName:0521c5d3-62eb-4ea7-b2e4-a2da4244d291 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:16.304557513 +0000 UTC m=+3.100772965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs") pod "network-metrics-daemon-9dbrx" (UID: "0521c5d3-62eb-4ea7-b2e4-a2da4244d291") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:15.404986 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.404957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:15.405127 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.405100 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:18:15.405127 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.405116 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:18:15.405127 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.405128 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jrxkn for pod openshift-network-diagnostics/network-check-target-cmpvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:15.405241 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.405174 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn podName:6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d nodeName:}" failed. No retries permitted until 2026-09-10 22:18:16.405162917 +0000 UTC m=+3.201378389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrxkn" (UniqueName: "kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn") pod "network-check-target-cmpvq" (UID: "6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:15.636214 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.636118 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 22:13:14 +0000 UTC" deadline="2028-04-04 00:13:15.610899056 +0000 UTC" Sep 10 22:18:15.636214 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.636172 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13705h54m59.974730363s" Sep 10 22:18:15.744010 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.743977 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:15.744183 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.744122 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:15.753832 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.752851 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" event={"ID":"9e300852a2fc99e2c8b674aec6bd8a96","Type":"ContainerStarted","Data":"38daaa33b24b4c6841d2746fe560162e666d3d98abfdf753bd6829e16d722960"} Sep 10 22:18:15.776024 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.775993 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" event={"ID":"be714b5e0febb4494d9a83009ae5a158","Type":"ContainerStarted","Data":"40cb9a062f7aebefacd8c908cd78ad4022ffe1932e7fb2d1168203198a944f3b"} Sep 10 22:18:15.792922 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.792891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"951430e9770798b164e99ab3bd2c62ff65bed732fcdeed7e27ddac6e4693fc21"} Sep 10 22:18:15.796347 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.796286 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" event={"ID":"a936c932-4a9f-4df8-8a50-3cd336664bff","Type":"ContainerStarted","Data":"d07f7042b5a9335b28ffb9c83c9e8c1d44b0dedcef37eddc0b9b9c257c1b5882"} Sep 10 22:18:15.814837 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.814792 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerStarted","Data":"4105357b3db92d3c47f32087c582f36e34b4e92bb2a104025e128cab9756b90d"} Sep 10 22:18:15.825503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.825477 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f595d" event={"ID":"c356678a-1be3-4972-901f-d1b445f890d1","Type":"ContainerStarted","Data":"831ce1587b866e6acb0af52004ad07dc796cf0ab70b06ed7bb18d256d1070cd2"} Sep 10 22:18:15.828359 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.828336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rstb7" event={"ID":"57467133-81f3-434e-ae9f-1f07cd84ceba","Type":"ContainerStarted","Data":"ef7904b85787da024e97e448707d0c83f63f91af4946b7b7fae422f73b857fc8"} Sep 10 22:18:15.835826 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.835782 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pbf7c" event={"ID":"147108fb-2bd0-4022-9d0c-1fae37ad557e","Type":"ContainerStarted","Data":"c8fb6541911663d97932ecdfd55e349dec2407e77f2b909af6adbfd0603dd8ca"} Sep 10 22:18:15.847006 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.846983 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fcrdb" event={"ID":"5308bd19-fb64-46c0-95aa-db0504e62f76","Type":"ContainerStarted","Data":"5ae1258c21828894110b72d68701166c817e3eb1137da5d3040f23e0d5dc39dd"} Sep 10 22:18:15.851162 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:15.849322 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53f547a13d43a4e4789407df692a7707f1a00f37794bf7afc92db6fdc85929a6\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-fcrdb" podUID="5308bd19-fb64-46c0-95aa-db0504e62f76" Sep 10 22:18:15.851216 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.851198 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-scrkj" event={"ID":"d2b6e10a-8490-4ef2-9146-601fb419ad66","Type":"ContainerStarted","Data":"6f417d77298c073e9a83c96517395743f541613a5875c7dd15d86b9dd426717c"} Sep 10 22:18:15.856865 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:15.855892 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nj85g" event={"ID":"63da62fc-f1bc-49e9-b2ff-0a39cf81331f","Type":"ContainerStarted","Data":"b0fd46c81dd4e76a110c98745c966faa155627cc3ae79b8802f9ccaaf3486ca3"} Sep 10 22:18:16.127037 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.127001 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:18:16.135502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.135482 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 22:18:16.311209 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.311165 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:16.311376 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.311250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:16.311376 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.311357 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:16.311481 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.311409 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs podName:0521c5d3-62eb-4ea7-b2e4-a2da4244d291 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:18.311394124 +0000 UTC m=+5.107609591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs") pod "network-metrics-daemon-9dbrx" (UID: "0521c5d3-62eb-4ea7-b2e4-a2da4244d291") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:16.311785 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.311765 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:16.311911 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.311834 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret podName:d0d3d831-2afd-4647-9c64-22c55eccf863 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:18.311800657 +0000 UTC m=+5.108016124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret") pod "global-pull-secret-syncer-g7tpk" (UID: "d0d3d831-2afd-4647-9c64-22c55eccf863") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:16.412532 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.412501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:16.412659 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.412651 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:18:16.412719 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.412670 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:18:16.412719 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.412681 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jrxkn for pod openshift-network-diagnostics/network-check-target-cmpvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:16.412845 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.412733 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn podName:6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d nodeName:}" failed. No retries permitted until 2026-09-10 22:18:18.412715972 +0000 UTC m=+5.208931443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrxkn" (UniqueName: "kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn") pod "network-check-target-cmpvq" (UID: "6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:16.636567 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.636446 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 22:13:14 +0000 UTC" deadline="2028-05-29 17:38:38.30229684 +0000 UTC" Sep 10 22:18:16.636567 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.636474 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15043h20m21.665826209s" Sep 10 22:18:16.743108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.743031 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:16.743255 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.743190 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:16.743562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:16.743543 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:16.743661 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:16.743623 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:17.745894 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:17.745865 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:17.746320 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:17.745997 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:18.329795 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:18.329759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:18.329968 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:18.329847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:18.329968 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.329944 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:18.329968 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.329945 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:18.330105 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.329999 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs podName:0521c5d3-62eb-4ea7-b2e4-a2da4244d291 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:22.329985624 +0000 UTC m=+9.126201079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs") pod "network-metrics-daemon-9dbrx" (UID: "0521c5d3-62eb-4ea7-b2e4-a2da4244d291") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:18.330105 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.330012 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret podName:d0d3d831-2afd-4647-9c64-22c55eccf863 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:22.330006318 +0000 UTC m=+9.126221773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret") pod "global-pull-secret-syncer-g7tpk" (UID: "d0d3d831-2afd-4647-9c64-22c55eccf863") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:18.430973 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:18.430932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:18.431153 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.431115 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:18:18.431153 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.431138 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:18:18.431153 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.431149 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jrxkn for pod openshift-network-diagnostics/network-check-target-cmpvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:18.431300 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.431219 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn podName:6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d nodeName:}" failed. No retries permitted until 2026-09-10 22:18:22.43120148 +0000 UTC m=+9.227416952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrxkn" (UniqueName: "kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn") pod "network-check-target-cmpvq" (UID: "6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:18.743540 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:18.743470 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:18.743696 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:18.743477 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:18.743696 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.743572 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:18.743696 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:18.743678 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:19.744837 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:19.744789 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:19.745270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:19.744937 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:20.742726 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:20.742697 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:20.742878 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:20.742806 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:20.742878 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:20.742841 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:20.742960 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:20.742935 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:21.743369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:21.742939 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:21.743369 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:21.743077 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:22.357025 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:22.356988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:22.357162 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:22.357040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:22.357162 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.357145 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:22.357162 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.357145 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:22.357302 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.357234 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret podName:d0d3d831-2afd-4647-9c64-22c55eccf863 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:30.357219668 +0000 UTC m=+17.153435140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret") pod "global-pull-secret-syncer-g7tpk" (UID: "d0d3d831-2afd-4647-9c64-22c55eccf863") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:22.357302 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.357257 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs podName:0521c5d3-62eb-4ea7-b2e4-a2da4244d291 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:30.357246846 +0000 UTC m=+17.153462300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs") pod "network-metrics-daemon-9dbrx" (UID: "0521c5d3-62eb-4ea7-b2e4-a2da4244d291") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:22.457993 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:22.457901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:22.458134 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.458039 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:18:22.458134 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.458055 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:18:22.458134 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.458083 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jrxkn for pod openshift-network-diagnostics/network-check-target-cmpvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:22.458265 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.458138 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn podName:6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d nodeName:}" failed. No retries permitted until 2026-09-10 22:18:30.458123604 +0000 UTC m=+17.254339063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrxkn" (UniqueName: "kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn") pod "network-check-target-cmpvq" (UID: "6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:22.743870 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:22.743290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:22.743870 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.743415 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:22.743870 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:22.743290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:22.743870 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:22.743770 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:23.744564 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:23.744104 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:23.744564 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:23.744226 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:24.743288 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:24.743252 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:24.743288 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:24.743296 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:24.743506 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:24.743390 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:24.743560 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:24.743511 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:25.743554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.743409 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:25.744232 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:25.743670 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:25.875758 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.875513 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-scrkj" event={"ID":"d2b6e10a-8490-4ef2-9146-601fb419ad66","Type":"ContainerStarted","Data":"c38a0b28ea2dd1a15e64ebbef2437fe4ce9b9244131ee5a1822561acae1d0802"} Sep 10 22:18:25.877337 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.877304 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nj85g" event={"ID":"63da62fc-f1bc-49e9-b2ff-0a39cf81331f","Type":"ContainerStarted","Data":"cdaa52f399c1d1c613ec25c57a477dc12ae1699decfc35bddf1614614a781c8a"} Sep 10 22:18:25.879153 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.879112 2577 generic.go:356] "Generic (PLEG): container finished" podID="9e300852a2fc99e2c8b674aec6bd8a96" containerID="2f6671c44bbeec0f5832b7d97e3f7f3931501778d1228fcf76e5d97771cc982d" exitCode=0 Sep 10 22:18:25.879272 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.879200 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" event={"ID":"9e300852a2fc99e2c8b674aec6bd8a96","Type":"ContainerDied","Data":"2f6671c44bbeec0f5832b7d97e3f7f3931501778d1228fcf76e5d97771cc982d"} Sep 10 22:18:25.880896 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.880873 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" event={"ID":"be714b5e0febb4494d9a83009ae5a158","Type":"ContainerStarted","Data":"5b6d76df0c18078827454c2387c454ebc32fd7e9b54b45b3640ada74e6997c3b"} Sep 10 22:18:25.882622 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.882599 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" event={"ID":"a936c932-4a9f-4df8-8a50-3cd336664bff","Type":"ContainerStarted","Data":"776b09e82acf2d4d37aeeef76fa9cef9e28bf95b24d3a8f67632e933e5b7c8d5"} Sep 10 22:18:25.884282 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.884263 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerStarted","Data":"5b8d597bca24743891a7a2b97ac51d1042c1a4bcbbd0299487e52b7ceadf87ee"} Sep 10 22:18:25.886163 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.886141 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f595d" event={"ID":"c356678a-1be3-4972-901f-d1b445f890d1","Type":"ContainerStarted","Data":"3d210fb3f41cd155c6d949c441157da0fd9dcacbc965df7bcf4843ec99d7d356"} Sep 10 22:18:25.887736 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.887713 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rstb7" event={"ID":"57467133-81f3-434e-ae9f-1f07cd84ceba","Type":"ContainerStarted","Data":"a4c88f070e4ad7f7edb5cdc61337af83f549c4931f6c4734dc095d89c621ea67"} Sep 10 22:18:25.890505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.890465 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-scrkj" podStartSLOduration=2.941135811 podStartE2EDuration="12.890453725s" totalImagesPullingTime="9.949317914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:25.889988536 +0000 UTC m=+12.686204014" watchObservedRunningTime="2026-09-10 22:18:25.890453725 +0000 UTC m=+12.686669202" Sep 10 22:18:25.915213 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.915170 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-235.ec2.internal" podStartSLOduration=11.915154678 podStartE2EDuration="11.915154678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:25.914619665 +0000 UTC m=+12.710835145" watchObservedRunningTime="2026-09-10 22:18:25.915154678 +0000 UTC m=+12.711370158" Sep 10 22:18:25.927606 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.927568 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rstb7" podStartSLOduration=2.914481628 podStartE2EDuration="12.927552309s" totalImagesPullingTime="10.013070681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:25.926938935 +0000 UTC m=+12.723154412" watchObservedRunningTime="2026-09-10 22:18:25.927552309 +0000 UTC m=+12.723767839" Sep 10 22:18:25.954766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.954680 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f595d" podStartSLOduration=2.916620499 podStartE2EDuration="12.954667319s" totalImagesPullingTime="10.03804682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:25.954291791 +0000 UTC m=+12.750507271" watchObservedRunningTime="2026-09-10 22:18:25.954667319 +0000 UTC m=+12.750882795" Sep 10 22:18:25.976791 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:25.976755 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nj85g" podStartSLOduration=2.974921692 podStartE2EDuration="12.97673813s" totalImagesPullingTime="10.001816438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:25.976534699 +0000 UTC m=+12.772750177" watchObservedRunningTime="2026-09-10 22:18:25.97673813 +0000 UTC m=+12.772953609" Sep 10 22:18:26.742802 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:26.742767 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:26.742980 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:26.742900 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:26.743059 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:26.743025 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:26.743148 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:26.743127 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:26.891312 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:26.891273 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pbf7c" event={"ID":"147108fb-2bd0-4022-9d0c-1fae37ad557e","Type":"ContainerStarted","Data":"233cc4c0ec4902bb65afab9a320062b622f3a472d8d5ffa453c0201ece723d79"} Sep 10 22:18:26.904988 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:26.904947 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pbf7c" podStartSLOduration=3.861929786 podStartE2EDuration="13.904932432s" totalImagesPullingTime="10.043002646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:26.904738312 +0000 UTC m=+13.700953790" watchObservedRunningTime="2026-09-10 22:18:26.904932432 +0000 UTC m=+13.701147909" Sep 10 22:18:27.097853 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:27.097799 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:27.098357 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:27.098338 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:27.742879 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:27.742850 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:27.743053 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:27.743000 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:27.893743 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:27.893702 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:27.894171 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:27.894085 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-scrkj" Sep 10 22:18:28.743350 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:28.743316 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:28.743546 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:28.743451 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:28.743546 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:28.743478 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:28.743675 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:28.743554 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:29.742904 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.742838 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:29.743396 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:29.742972 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:29.899334 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.899307 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" event={"ID":"9e300852a2fc99e2c8b674aec6bd8a96","Type":"ContainerStarted","Data":"9bbaf674a029b3b0f9f720455deabb1a5bffa8015e3e5153a84b7222eee7c4f4"} Sep 10 22:18:29.902231 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.902004 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"a87f3ac5323b95d248fe0fb2c5b6170733f4d8f8b3da75f69ea9194a1527e3bc"} Sep 10 22:18:29.902231 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.902091 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"d51ddbfcbcfd13cdc5059fef19843de91e2ec993007901a1426e9ddcb77bec61"} Sep 10 22:18:29.902231 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.902139 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"82a3be9935214c61ab1fb0d34217bab09748fe6f5307739be790f8f5342c9dae"} Sep 10 22:18:29.903743 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.903719 2577 generic.go:356] "Generic (PLEG): container finished" podID="079e30d5-078c-4982-981d-f4df3a17834b" containerID="5b8d597bca24743891a7a2b97ac51d1042c1a4bcbbd0299487e52b7ceadf87ee" exitCode=0 Sep 10 22:18:29.904141 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.904124 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerDied","Data":"5b8d597bca24743891a7a2b97ac51d1042c1a4bcbbd0299487e52b7ceadf87ee"} Sep 10 22:18:29.920314 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.918709 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-235.ec2.internal" podStartSLOduration=15.918692348 podStartE2EDuration="15.918692348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:29.918191725 +0000 UTC m=+16.714407203" watchObservedRunningTime="2026-09-10 22:18:29.918692348 +0000 UTC m=+16.714907827" Sep 10 22:18:29.999593 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:29.999536 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 22:18:30.420512 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.420476 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:30.420659 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.420530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:30.420659 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.420582 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:30.420659 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.420620 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:30.420659 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.420645 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs podName:0521c5d3-62eb-4ea7-b2e4-a2da4244d291 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.420629971 +0000 UTC m=+33.216845426 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs") pod "network-metrics-daemon-9dbrx" (UID: "0521c5d3-62eb-4ea7-b2e4-a2da4244d291") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:30.420659 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.420662 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret podName:d0d3d831-2afd-4647-9c64-22c55eccf863 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.420652221 +0000 UTC m=+33.216867680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret") pod "global-pull-secret-syncer-g7tpk" (UID: "d0d3d831-2afd-4647-9c64-22c55eccf863") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:30.521056 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.521019 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:30.521220 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.521192 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 22:18:30.521220 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.521213 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 22:18:30.521329 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.521227 2577 projected.go:196] Error preparing data for projected volume kube-api-access-jrxkn for pod openshift-network-diagnostics/network-check-target-cmpvq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:30.521329 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.521286 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn podName:6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.52126737 +0000 UTC m=+33.317482836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrxkn" (UniqueName: "kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn") pod "network-check-target-cmpvq" (UID: "6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 22:18:30.659827 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.659710 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T22:18:29.999551887Z","UUID":"7dd28312-788b-426f-87e7-3d456bd095d2","Handler":null,"Name":"","Endpoint":""} Sep 10 22:18:30.661359 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.661336 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 22:18:30.661485 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.661371 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 22:18:30.743302 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.743223 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:30.743302 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.743266 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:30.743887 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.743367 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:30.743887 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:30.743513 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:30.910727 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.910691 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"def8f89ff44e8aae6cc4016783c6f6161905a4b55a2fd7f8b3707eeebc15b343"} Sep 10 22:18:30.910929 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.910787 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"e7ecf560e2e6b4b4674d48fe4a54a2d99fb7eb0a4508767626aa9e1de344d8e2"} Sep 10 22:18:30.910929 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.910857 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"1944bc23476800be4772e84cdea1880ea8dc05da584c53936e4f242107b23f26"} Sep 10 22:18:30.914764 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:30.914691 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" event={"ID":"a936c932-4a9f-4df8-8a50-3cd336664bff","Type":"ContainerStarted","Data":"569d7a9604c062a9158b55891bd15184b0cab9397417336f2d62b3bc751c83cb"} Sep 10 22:18:31.743799 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:31.743766 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:31.744242 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:31.743906 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:31.919725 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:31.919234 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" event={"ID":"a936c932-4a9f-4df8-8a50-3cd336664bff","Type":"ContainerStarted","Data":"82ed7fdfab101e7d91df5232ffeed38d7334d2321605fd0e7c2a085125520b1d"} Sep 10 22:18:31.936083 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:31.936039 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hz54x" podStartSLOduration=3.6559801480000003 podStartE2EDuration="18.936022758s" totalImagesPullingTime="15.28004261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:31.935344649 +0000 UTC m=+18.731560126" watchObservedRunningTime="2026-09-10 22:18:31.936022758 +0000 UTC m=+18.732238229" Sep 10 22:18:32.742857 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:32.742810 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:32.743047 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:32.742934 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:32.743047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:32.743001 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:32.743153 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:32.743100 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:32.925762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:32.925713 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"66ccbfd368171d097873e0b20499a4ba724c5c16be64960a30f397c9be2487e9"} Sep 10 22:18:33.744959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:33.744924 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:33.745131 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:33.745036 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:33.745131 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:33.745097 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:33.745248 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:33.745226 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:34.743499 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:34.743467 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:34.743960 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:34.743599 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:35.743003 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:35.742964 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:35.743003 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:35.743001 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:35.743300 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:35.743082 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:35.743300 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:35.743218 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:36.743388 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:36.743350 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:36.743956 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:36.743482 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:37.747387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:37.746942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:37.747387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:37.746992 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:37.747387 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:37.747072 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:37.747387 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:37.747193 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:38.742985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:38.742949 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:38.743152 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:38.743056 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:39.743737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.743553 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:39.744164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.743574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:39.744164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:39.743838 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:39.744164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:39.743934 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:39.945318 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.945234 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" event={"ID":"80749e7c-a14b-4f64-ac75-c35eb2f5b359","Type":"ContainerStarted","Data":"305063e1c615849d68e9943760aab117150e69b838cbb6eb3d952a588efa70e1"} Sep 10 22:18:39.945504 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.945471 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:39.947227 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.947198 2577 generic.go:356] "Generic (PLEG): container finished" podID="079e30d5-078c-4982-981d-f4df3a17834b" containerID="03a1dd6360f4554a79ffdfe50e2a34c7e5610b6cf8cc397afb15a3a90437c184" exitCode=0 Sep 10 22:18:39.947342 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.947264 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerDied","Data":"03a1dd6360f4554a79ffdfe50e2a34c7e5610b6cf8cc397afb15a3a90437c184"} Sep 10 22:18:39.949036 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.949016 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fcrdb" event={"ID":"5308bd19-fb64-46c0-95aa-db0504e62f76","Type":"ContainerStarted","Data":"f04cf28a61245ab64b04c6c729d9182c0f390f8b3371ab962e9ed7e65754be66"} Sep 10 22:18:39.962084 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.962068 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:39.970805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:39.969557 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" podStartSLOduration=12.543063825 podStartE2EDuration="26.969541697s" totalImagesPullingTime="14.426477872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:39.968502155 +0000 UTC m=+26.764717633" watchObservedRunningTime="2026-09-10 22:18:39.969541697 +0000 UTC m=+26.765757175" Sep 10 22:18:40.026658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:40.026618 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fcrdb" podStartSLOduration=2.650978671 podStartE2EDuration="27.026605994s" totalImagesPullingTime="24.375627323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-10 22:18:40.026517249 +0000 UTC m=+26.822732727" watchObservedRunningTime="2026-09-10 22:18:40.026605994 +0000 UTC m=+26.822821515" Sep 10 22:18:40.743250 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:40.743169 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:40.743411 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:40.743291 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:40.951902 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:40.951878 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 22:18:40.952342 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:40.952322 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:40.974217 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:40.974050 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:41.338955 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.338925 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9dbrx"] Sep 10 22:18:41.339116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.339065 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:41.339188 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:41.339169 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:41.342444 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.342417 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmpvq"] Sep 10 22:18:41.342559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.342515 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:41.342635 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:41.342616 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:41.343019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.342996 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g7tpk"] Sep 10 22:18:41.343132 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.343084 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:41.343202 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:41.343185 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:41.955896 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.955865 2577 generic.go:356] "Generic (PLEG): container finished" podID="079e30d5-078c-4982-981d-f4df3a17834b" containerID="69498876aee73fc034082f334ad62fa1cf5d54e0f152c7b4c9ada7a219794aec" exitCode=0 Sep 10 22:18:41.956329 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.955932 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerDied","Data":"69498876aee73fc034082f334ad62fa1cf5d54e0f152c7b4c9ada7a219794aec"} Sep 10 22:18:41.956329 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:41.956029 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 22:18:42.743439 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:42.743401 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:42.743594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:42.743401 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:42.743594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:42.743415 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:42.743594 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:42.743581 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:42.743766 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:42.743699 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:42.743876 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:42.743837 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:42.958421 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:42.958394 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 10 22:18:43.711103 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:43.711044 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:18:43.962463 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:43.962380 2577 generic.go:356] "Generic (PLEG): container finished" podID="079e30d5-078c-4982-981d-f4df3a17834b" containerID="9c1acf5d27cdcc5c2057e250fc88b32f9e0ed629835676b2807a0c19aef7010d" exitCode=0 Sep 10 22:18:43.962980 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:43.962452 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerDied","Data":"9c1acf5d27cdcc5c2057e250fc88b32f9e0ed629835676b2807a0c19aef7010d"} Sep 10 22:18:44.743778 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:44.743748 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:44.743947 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:44.743879 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9dbrx" podUID="0521c5d3-62eb-4ea7-b2e4-a2da4244d291" Sep 10 22:18:44.743992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:44.743970 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:44.744079 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:44.744061 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g7tpk" podUID="d0d3d831-2afd-4647-9c64-22c55eccf863" Sep 10 22:18:44.744116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:44.744110 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:44.744173 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:44.744161 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmpvq" podUID="6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d" Sep 10 22:18:44.979356 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:44.979327 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-235.ec2.internal" event="NodeReady" Sep 10 22:18:44.979763 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:44.979458 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 22:18:45.020756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.020682 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-jv4bp"] Sep 10 22:18:45.037723 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.037699 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c7d7d47c8-2k8vr"] Sep 10 22:18:45.037866 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.037836 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.042110 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.042066 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 10 22:18:45.043099 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.042679 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.043099 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.042940 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ddkcr\"" Sep 10 22:18:45.045132 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.044796 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.045132 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.044795 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 10 22:18:45.051941 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.051919 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 10 22:18:45.056363 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.056321 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v"] Sep 10 22:18:45.056656 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.056634 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.059069 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.059051 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 22:18:45.059206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.059060 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 22:18:45.059290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.059131 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6vhns\"" Sep 10 22:18:45.061805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.061385 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 22:18:45.068221 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.068204 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m"] Sep 10 22:18:45.068330 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.068317 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" Sep 10 22:18:45.070457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.070437 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sv69c\"" Sep 10 22:18:45.070567 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.070460 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 22:18:45.070567 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.070556 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.070676 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.070579 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.082728 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.082711 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw"] Sep 10 22:18:45.082839 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.082806 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:45.084927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.084910 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 10 22:18:45.085100 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.085088 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.085175 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.085128 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.085329 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.085314 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pflv9\"" Sep 10 22:18:45.098988 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.098959 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v"] Sep 10 22:18:45.098988 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.098991 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-jv4bp"] Sep 10 22:18:45.099127 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.099005 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw"] Sep 10 22:18:45.099127 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.099021 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m"] Sep 10 22:18:45.099127 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.099042 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6lz6l"] Sep 10 22:18:45.099127 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.099083 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.101674 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.101658 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 10 22:18:45.101757 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.101664 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.105960 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.102776 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zm8fb\"" Sep 10 22:18:45.106040 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.102769 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.107331 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.107313 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 10 22:18:45.114158 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.114139 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-866j5"] Sep 10 22:18:45.114344 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.114223 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.116490 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.116453 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 22:18:45.116588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.116495 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 22:18:45.116771 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.116751 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bjgt9\"" Sep 10 22:18:45.124374 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.124355 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6cf6d56fcd-xqxl8"] Sep 10 22:18:45.124526 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.124510 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.127025 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.127008 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.127025 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.127034 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.127187 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.127065 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-m8mcb\"" Sep 10 22:18:45.127187 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.127010 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 10 22:18:45.127515 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.127496 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 10 22:18:45.131552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.131552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-trusted-ca\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.131552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131378 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-installation-pull-secrets\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.131552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-certificates\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.131552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131448 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/099a0642-5254-4571-8da9-e6cce52ff27f-serving-cert\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.131552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131487 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-image-registry-private-configuration\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.131552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099a0642-5254-4571-8da9-e6cce52ff27f-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131574 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/099a0642-5254-4571-8da9-e6cce52ff27f-snapshots\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131628 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-ca-trust-extracted\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr4fk\" (UniqueName: \"kubernetes.io/projected/099a0642-5254-4571-8da9-e6cce52ff27f-kube-api-access-zr4fk\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131693 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-bound-sa-token\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131718 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x754l\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-kube-api-access-x754l\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131767 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v"] Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131776 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099a0642-5254-4571-8da9-e6cce52ff27f-service-ca-bundle\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m7jx\" (UniqueName: \"kubernetes.io/projected/75d82860-197d-4819-9059-222e9c024ddf-kube-api-access-7m7jx\") pod \"volume-data-source-validator-86695b7cc7-frh8v\" (UID: \"75d82860-197d-4819-9059-222e9c024ddf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131845 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131878 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/099a0642-5254-4571-8da9-e6cce52ff27f-tmp\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.132001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.131981 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 10 22:18:45.134026 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.134008 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 10 22:18:45.134137 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.134044 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 10 22:18:45.134262 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.134243 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 10 22:18:45.134338 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.134313 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.134548 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.134530 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sn269\"" Sep 10 22:18:45.134644 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.134635 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 10 22:18:45.134697 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.134655 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.142100 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.142085 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2"] Sep 10 22:18:45.142201 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.142186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:45.144140 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.144121 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7529d\"" Sep 10 22:18:45.144230 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.144130 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 22:18:45.144368 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.144351 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 22:18:45.152289 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.152269 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n9pms"] Sep 10 22:18:45.152453 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.152435 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.154448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.154426 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-247r7\"" Sep 10 22:18:45.154630 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.154612 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 10 22:18:45.154736 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.154660 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.154798 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.154787 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.154871 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.154862 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 10 22:18:45.162481 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.162463 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf"] Sep 10 22:18:45.162587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.162569 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.164692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.164677 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fp646\"" Sep 10 22:18:45.164965 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.164721 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.165069 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.165010 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.165177 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.165155 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 22:18:45.172789 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.172767 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-q44g9"] Sep 10 22:18:45.172914 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.172897 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.174959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.174937 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 10 22:18:45.175054 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.175033 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.175107 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.174938 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 10 22:18:45.175147 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.174941 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.175363 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.175347 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-86gdx\"" Sep 10 22:18:45.182970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.182952 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c7d7d47c8-2k8vr"] Sep 10 22:18:45.183070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.182988 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-q44g9"] Sep 10 22:18:45.183070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183010 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf"] Sep 10 22:18:45.183070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183029 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2"] Sep 10 22:18:45.183070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183048 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-866j5"] Sep 10 22:18:45.183070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183066 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v"] Sep 10 22:18:45.183276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183085 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6cf6d56fcd-xqxl8"] Sep 10 22:18:45.183276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183098 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" Sep 10 22:18:45.183276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183103 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6lz6l"] Sep 10 22:18:45.183276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.183255 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n9pms"] Sep 10 22:18:45.185158 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.185139 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 22:18:45.185283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.185264 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 22:18:45.185344 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.185265 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2g8jr\"" Sep 10 22:18:45.232940 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.232899 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-image-registry-private-configuration\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.233070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.232945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84a7c959-b185-469d-8a42-30171c664a31-serving-cert\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.233070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.232973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:45.233070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.232997 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-serving-cert\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.233252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233123 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfx4f\" (UniqueName: \"kubernetes.io/projected/13205e70-75c2-4340-a014-0f3d44963cbd-kube-api-access-pfx4f\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.233252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233152 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-tmp-dir\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.233252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q7cg\" (UniqueName: \"kubernetes.io/projected/82c80596-0e41-4e7b-9f6a-c27da394a286-kube-api-access-8q7cg\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:45.233252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8z8m\" (UniqueName: \"kubernetes.io/projected/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-kube-api-access-q8z8m\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.233252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d6f97834-3301-409b-a2ea-a6fd959d3068-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233269 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13205e70-75c2-4340-a014-0f3d44963cbd-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktkb6\" (UniqueName: \"kubernetes.io/projected/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-kube-api-access-ktkb6\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233329 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-bound-sa-token\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233353 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x754l\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-kube-api-access-x754l\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7dbz\" (UniqueName: \"kubernetes.io/projected/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-kube-api-access-t7dbz\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233407 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef748efe-3001-4cdf-b168-2b215ee1cd9a-cert\") pod \"ingress-canary-n9pms\" (UID: \"ef748efe-3001-4cdf-b168-2b215ee1cd9a\") " pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233430 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099a0642-5254-4571-8da9-e6cce52ff27f-service-ca-bundle\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.233455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233453 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mmcs\" (UniqueName: \"kubernetes.io/projected/ef748efe-3001-4cdf-b168-2b215ee1cd9a-kube-api-access-8mmcs\") pod \"ingress-canary-n9pms\" (UID: \"ef748efe-3001-4cdf-b168-2b215ee1cd9a\") " pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233480 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-config\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233503 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-config-volume\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233527 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099a0642-5254-4571-8da9-e6cce52ff27f-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-trusted-ca\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-installation-pull-secrets\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-certificates\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-default-certificate\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233699 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233747 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/099a0642-5254-4571-8da9-e6cce52ff27f-snapshots\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8s2n\" (UniqueName: \"kubernetes.io/projected/84a7c959-b185-469d-8a42-30171c664a31-kube-api-access-f8s2n\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:45.233841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233837 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13205e70-75c2-4340-a014-0f3d44963cbd-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a7c959-b185-469d-8a42-30171c664a31-trusted-ca\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233929 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-ca-trust-extracted\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233955 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr4fk\" (UniqueName: \"kubernetes.io/projected/099a0642-5254-4571-8da9-e6cce52ff27f-kube-api-access-zr4fk\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.233984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m7jx\" (UniqueName: \"kubernetes.io/projected/75d82860-197d-4819-9059-222e9c024ddf-kube-api-access-7m7jx\") pod \"volume-data-source-validator-86695b7cc7-frh8v\" (UID: \"75d82860-197d-4819-9059-222e9c024ddf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.234008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.234037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/099a0642-5254-4571-8da9-e6cce52ff27f-tmp\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.234060 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.234111 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-stats-auth\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.234136 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhzz\" (UniqueName: \"kubernetes.io/projected/d6f97834-3301-409b-a2ea-a6fd959d3068-kube-api-access-hvhzz\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.234171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84a7c959-b185-469d-8a42-30171c664a31-config\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.234418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.234201 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/099a0642-5254-4571-8da9-e6cce52ff27f-serving-cert\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.235101 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.235125 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c7d7d47c8-2k8vr: secret "image-registry-tls" not found Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.235191 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls podName:5378dc3b-6b96-428f-abf9-05d5c9ae9e6a nodeName:}" failed. No retries permitted until 2026-09-10 22:18:45.73517222 +0000 UTC m=+32.531387698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls") pod "image-registry-5c7d7d47c8-2k8vr" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a") : secret "image-registry-tls" not found Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.235548 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-certificates\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.235606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/099a0642-5254-4571-8da9-e6cce52ff27f-snapshots\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.235944 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-ca-trust-extracted\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.235953 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/099a0642-5254-4571-8da9-e6cce52ff27f-tmp\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.236239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.236034 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099a0642-5254-4571-8da9-e6cce52ff27f-service-ca-bundle\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.236528 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.236436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/099a0642-5254-4571-8da9-e6cce52ff27f-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.236738 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.236715 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-trusted-ca\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.237766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.237709 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/099a0642-5254-4571-8da9-e6cce52ff27f-serving-cert\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.237766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.237726 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-image-registry-private-configuration\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.238882 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.238714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-installation-pull-secrets\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.245556 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.245533 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-bound-sa-token\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.246312 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.246290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x754l\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-kube-api-access-x754l\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.246546 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.246517 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m7jx\" (UniqueName: \"kubernetes.io/projected/75d82860-197d-4819-9059-222e9c024ddf-kube-api-access-7m7jx\") pod \"volume-data-source-validator-86695b7cc7-frh8v\" (UID: \"75d82860-197d-4819-9059-222e9c024ddf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" Sep 10 22:18:45.246990 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.246962 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr4fk\" (UniqueName: \"kubernetes.io/projected/099a0642-5254-4571-8da9-e6cce52ff27f-kube-api-access-zr4fk\") pod \"insights-operator-5844b856db-jv4bp\" (UID: \"099a0642-5254-4571-8da9-e6cce52ff27f\") " pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.335462 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335425 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-stats-auth\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhzz\" (UniqueName: \"kubernetes.io/projected/d6f97834-3301-409b-a2ea-a6fd959d3068-kube-api-access-hvhzz\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335541 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84a7c959-b185-469d-8a42-30171c664a31-config\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84a7c959-b185-469d-8a42-30171c664a31-serving-cert\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.335583 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335589 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335614 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-serving-cert\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.335649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfx4f\" (UniqueName: \"kubernetes.io/projected/13205e70-75c2-4340-a014-0f3d44963cbd-kube-api-access-pfx4f\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.335659 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls podName:2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:45.835639134 +0000 UTC m=+32.631854609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls") pod "dns-default-6lz6l" (UID: "2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20") : secret "dns-default-metrics-tls" not found Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-tmp-dir\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q7cg\" (UniqueName: \"kubernetes.io/projected/82c80596-0e41-4e7b-9f6a-c27da394a286-kube-api-access-8q7cg\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335744 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8z8m\" (UniqueName: \"kubernetes.io/projected/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-kube-api-access-q8z8m\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335775 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29wqn\" (UniqueName: \"kubernetes.io/projected/367f7843-3bbe-4c67-8a5c-cc7a03f5b123-kube-api-access-29wqn\") pod \"network-check-source-6f8b768455-q44g9\" (UID: \"367f7843-3bbe-4c67-8a5c-cc7a03f5b123\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.335803 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335842 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d6f97834-3301-409b-a2ea-a6fd959d3068-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13205e70-75c2-4340-a014-0f3d44963cbd-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.335900 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert podName:ef9fa3f2-f8a4-40e2-aa74-de5824a3b356 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:45.835880741 +0000 UTC m=+32.632096209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rlb5v" (UID: "ef9fa3f2-f8a4-40e2-aa74-de5824a3b356") : secret "networking-console-plugin-cert" not found Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335925 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktkb6\" (UniqueName: \"kubernetes.io/projected/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-kube-api-access-ktkb6\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335967 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7dbz\" (UniqueName: \"kubernetes.io/projected/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-kube-api-access-t7dbz\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.335994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef748efe-3001-4cdf-b168-2b215ee1cd9a-cert\") pod \"ingress-canary-n9pms\" (UID: \"ef748efe-3001-4cdf-b168-2b215ee1cd9a\") " pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336021 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mmcs\" (UniqueName: \"kubernetes.io/projected/ef748efe-3001-4cdf-b168-2b215ee1cd9a-kube-api-access-8mmcs\") pod \"ingress-canary-n9pms\" (UID: \"ef748efe-3001-4cdf-b168-2b215ee1cd9a\") " pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336054 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-config\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.336206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-config-volume\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-default-certificate\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336231 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8s2n\" (UniqueName: \"kubernetes.io/projected/84a7c959-b185-469d-8a42-30171c664a31-kube-api-access-f8s2n\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13205e70-75c2-4340-a014-0f3d44963cbd-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336515 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a7c959-b185-469d-8a42-30171c664a31-trusted-ca\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.336583 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.336710 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.336756 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:45.836741224 +0000 UTC m=+32.632956697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : secret "router-metrics-certs-default" not found Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.337020 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/84a7c959-b185-469d-8a42-30171c664a31-config\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.337146 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:45.337222 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.337206 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls podName:d6f97834-3301-409b-a2ea-a6fd959d3068 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:45.837191898 +0000 UTC m=+32.633407358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4pkxw" (UID: "d6f97834-3301-409b-a2ea-a6fd959d3068") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:45.337956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.337425 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-config-volume\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.337956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.337599 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:45.337956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.337694 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13205e70-75c2-4340-a014-0f3d44963cbd-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.338317 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.338285 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-tmp-dir\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.338317 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.338313 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-config\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.338465 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.338396 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 22:18:45.338465 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.338440 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls podName:82c80596-0e41-4e7b-9f6a-c27da394a286 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:45.838424379 +0000 UTC m=+32.634639848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s5v9m" (UID: "82c80596-0e41-4e7b-9f6a-c27da394a286") : secret "samples-operator-tls" not found Sep 10 22:18:45.338583 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.338493 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84a7c959-b185-469d-8a42-30171c664a31-trusted-ca\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.338672 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.338653 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-stats-auth\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.338724 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.338656 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:45.838640086 +0000 UTC m=+32.634855573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : configmap references non-existent config key: service-ca.crt Sep 10 22:18:45.339040 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.339020 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-serving-cert\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.339261 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.339165 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d6f97834-3301-409b-a2ea-a6fd959d3068-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.339869 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.339745 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef748efe-3001-4cdf-b168-2b215ee1cd9a-cert\") pod \"ingress-canary-n9pms\" (UID: \"ef748efe-3001-4cdf-b168-2b215ee1cd9a\") " pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.341217 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.341177 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-default-certificate\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.341613 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.341589 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/84a7c959-b185-469d-8a42-30171c664a31-serving-cert\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.346071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.344999 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfx4f\" (UniqueName: \"kubernetes.io/projected/13205e70-75c2-4340-a014-0f3d44963cbd-kube-api-access-pfx4f\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.346071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.345501 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktkb6\" (UniqueName: \"kubernetes.io/projected/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-kube-api-access-ktkb6\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.346071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.345516 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhzz\" (UniqueName: \"kubernetes.io/projected/d6f97834-3301-409b-a2ea-a6fd959d3068-kube-api-access-hvhzz\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.346071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.346032 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8s2n\" (UniqueName: \"kubernetes.io/projected/84a7c959-b185-469d-8a42-30171c664a31-kube-api-access-f8s2n\") pod \"console-operator-696884fb9c-866j5\" (UID: \"84a7c959-b185-469d-8a42-30171c664a31\") " pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.346800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.346757 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13205e70-75c2-4340-a014-0f3d44963cbd-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-ndxsf\" (UID: \"13205e70-75c2-4340-a014-0f3d44963cbd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.346920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.346883 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q7cg\" (UniqueName: \"kubernetes.io/projected/82c80596-0e41-4e7b-9f6a-c27da394a286-kube-api-access-8q7cg\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:45.346920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.346904 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8z8m\" (UniqueName: \"kubernetes.io/projected/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-kube-api-access-q8z8m\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.347128 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.347107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7dbz\" (UniqueName: \"kubernetes.io/projected/e9f0338c-f42f-4873-8ebe-7ae3d7541cda-kube-api-access-t7dbz\") pod \"service-ca-operator-cf7c776b7-4gsm2\" (UID: \"e9f0338c-f42f-4873-8ebe-7ae3d7541cda\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.347277 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.347257 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mmcs\" (UniqueName: \"kubernetes.io/projected/ef748efe-3001-4cdf-b168-2b215ee1cd9a-kube-api-access-8mmcs\") pod \"ingress-canary-n9pms\" (UID: \"ef748efe-3001-4cdf-b168-2b215ee1cd9a\") " pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.355904 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.355883 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-jv4bp" Sep 10 22:18:45.377009 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.376985 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" Sep 10 22:18:45.436019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.435989 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:18:45.437309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.437290 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29wqn\" (UniqueName: \"kubernetes.io/projected/367f7843-3bbe-4c67-8a5c-cc7a03f5b123-kube-api-access-29wqn\") pod \"network-check-source-6f8b768455-q44g9\" (UID: \"367f7843-3bbe-4c67-8a5c-cc7a03f5b123\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" Sep 10 22:18:45.444914 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.444894 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29wqn\" (UniqueName: \"kubernetes.io/projected/367f7843-3bbe-4c67-8a5c-cc7a03f5b123-kube-api-access-29wqn\") pod \"network-check-source-6f8b768455-q44g9\" (UID: \"367f7843-3bbe-4c67-8a5c-cc7a03f5b123\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" Sep 10 22:18:45.461397 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.461365 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" Sep 10 22:18:45.472539 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.472507 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n9pms" Sep 10 22:18:45.482562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.482544 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" Sep 10 22:18:45.494786 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.494761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" Sep 10 22:18:45.740364 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.740337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:45.740520 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.740487 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:18:45.740520 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.740508 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c7d7d47c8-2k8vr: secret "image-registry-tls" not found Sep 10 22:18:45.740605 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.740574 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls podName:5378dc3b-6b96-428f-abf9-05d5c9ae9e6a nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.740554109 +0000 UTC m=+33.536769569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls") pod "image-registry-5c7d7d47c8-2k8vr" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a") : secret "image-registry-tls" not found Sep 10 22:18:45.774644 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.774619 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2"] Sep 10 22:18:45.777333 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.777311 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-jv4bp"] Sep 10 22:18:45.780357 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.780334 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v"] Sep 10 22:18:45.783427 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.783404 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-866j5"] Sep 10 22:18:45.793014 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:45.792939 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d82860_197d_4819_9059_222e9c024ddf.slice/crio-c80cb0c63472eb72985aae0f8401798435f51e553030e28d07becf41e10a574b WatchSource:0}: Error finding container c80cb0c63472eb72985aae0f8401798435f51e553030e28d07becf41e10a574b: Status 404 returned error can't find the container with id c80cb0c63472eb72985aae0f8401798435f51e553030e28d07becf41e10a574b Sep 10 22:18:45.793609 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:45.793176 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a7c959_b185_469d_8a42_30171c664a31.slice/crio-a19042cabbee15d784d7d34cb206abbc56cd1724f4ccd6d13f629f65672fe74b WatchSource:0}: Error finding container a19042cabbee15d784d7d34cb206abbc56cd1724f4ccd6d13f629f65672fe74b: Status 404 returned error can't find the container with id a19042cabbee15d784d7d34cb206abbc56cd1724f4ccd6d13f629f65672fe74b Sep 10 22:18:45.823553 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.823026 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n9pms"] Sep 10 22:18:45.825354 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:45.825329 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef748efe_3001_4cdf_b168_2b215ee1cd9a.slice/crio-1ac63f05c50ed5a7a38d36e973d6e50c45c8b80ce72cd644bc9663b2912e6d28 WatchSource:0}: Error finding container 1ac63f05c50ed5a7a38d36e973d6e50c45c8b80ce72cd644bc9663b2912e6d28: Status 404 returned error can't find the container with id 1ac63f05c50ed5a7a38d36e973d6e50c45c8b80ce72cd644bc9663b2912e6d28 Sep 10 22:18:45.834651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.834629 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf"] Sep 10 22:18:45.838371 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:45.838335 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13205e70_75c2_4340_a014_0f3d44963cbd.slice/crio-d8228e63eb2012db1043d8f7ac21df8c410bf019e26e1df9e6de9e2598e08929 WatchSource:0}: Error finding container d8228e63eb2012db1043d8f7ac21df8c410bf019e26e1df9e6de9e2598e08929: Status 404 returned error can't find the container with id d8228e63eb2012db1043d8f7ac21df8c410bf019e26e1df9e6de9e2598e08929 Sep 10 22:18:45.840723 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.840702 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:45.840866 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.840847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:45.840941 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.840872 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 22:18:45.840941 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.840881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.840941 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.840910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:45.840941 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.840933 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert podName:ef9fa3f2-f8a4-40e2-aa74-de5824a3b356 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.840911187 +0000 UTC m=+33.637126649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rlb5v" (UID: "ef9fa3f2-f8a4-40e2-aa74-de5824a3b356") : secret "networking-console-plugin-cert" not found Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.840980 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.840981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841016 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls podName:d6f97834-3301-409b-a2ea-a6fd959d3068 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.841005639 +0000 UTC m=+33.637221094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4pkxw" (UID: "d6f97834-3301-409b-a2ea-a6fd959d3068") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.841029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841066 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841105 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.841091279 +0000 UTC m=+33.637306747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : secret "router-metrics-certs-default" not found Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841137 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:18:45.841164 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841168 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls podName:2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.841158376 +0000 UTC m=+33.637373831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls") pod "dns-default-6lz6l" (UID: "2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20") : secret "dns-default-metrics-tls" not found Sep 10 22:18:45.841550 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841179 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 22:18:45.841550 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841217 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls podName:82c80596-0e41-4e7b-9f6a-c27da394a286 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.841204616 +0000 UTC m=+33.637420074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s5v9m" (UID: "82c80596-0e41-4e7b-9f6a-c27da394a286") : secret "samples-operator-tls" not found Sep 10 22:18:45.841550 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:45.841241 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:46.841227726 +0000 UTC m=+33.637443181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : configmap references non-existent config key: service-ca.crt Sep 10 22:18:45.859668 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.859612 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-q44g9"] Sep 10 22:18:45.862342 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:45.862315 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367f7843_3bbe_4c67_8a5c_cc7a03f5b123.slice/crio-d144b9640a327db057988e8ed5766a237011b167b2b7a4c0f5dc8b738bfad7a2 WatchSource:0}: Error finding container d144b9640a327db057988e8ed5766a237011b167b2b7a4c0f5dc8b738bfad7a2: Status 404 returned error can't find the container with id d144b9640a327db057988e8ed5766a237011b167b2b7a4c0f5dc8b738bfad7a2 Sep 10 22:18:45.967369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.967334 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" event={"ID":"367f7843-3bbe-4c67-8a5c-cc7a03f5b123","Type":"ContainerStarted","Data":"d144b9640a327db057988e8ed5766a237011b167b2b7a4c0f5dc8b738bfad7a2"} Sep 10 22:18:45.968468 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.968443 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" event={"ID":"13205e70-75c2-4340-a014-0f3d44963cbd","Type":"ContainerStarted","Data":"d8228e63eb2012db1043d8f7ac21df8c410bf019e26e1df9e6de9e2598e08929"} Sep 10 22:18:45.969555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.969529 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-866j5" event={"ID":"84a7c959-b185-469d-8a42-30171c664a31","Type":"ContainerStarted","Data":"a19042cabbee15d784d7d34cb206abbc56cd1724f4ccd6d13f629f65672fe74b"} Sep 10 22:18:45.970609 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.970582 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" event={"ID":"e9f0338c-f42f-4873-8ebe-7ae3d7541cda","Type":"ContainerStarted","Data":"e42d1c4c60583860feb10a321436d16b93b1ba21c5d000f43cff65a067f5d4e5"} Sep 10 22:18:45.971588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.971572 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" event={"ID":"75d82860-197d-4819-9059-222e9c024ddf","Type":"ContainerStarted","Data":"c80cb0c63472eb72985aae0f8401798435f51e553030e28d07becf41e10a574b"} Sep 10 22:18:45.973091 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.973071 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jv4bp" event={"ID":"099a0642-5254-4571-8da9-e6cce52ff27f","Type":"ContainerStarted","Data":"f1729015e6d32f82aa10f67c9c2714bf7bebcbb2797f5a172a16e3eb9dc27c75"} Sep 10 22:18:45.974221 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:45.974202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n9pms" event={"ID":"ef748efe-3001-4cdf-b168-2b215ee1cd9a","Type":"ContainerStarted","Data":"1ac63f05c50ed5a7a38d36e973d6e50c45c8b80ce72cd644bc9663b2912e6d28"} Sep 10 22:18:46.446494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.446394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:46.447037 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.446565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:46.447037 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.446735 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:46.447037 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.446811 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret podName:d0d3d831-2afd-4647-9c64-22c55eccf863 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:18.446792057 +0000 UTC m=+65.243007531 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret") pod "global-pull-secret-syncer-g7tpk" (UID: "d0d3d831-2afd-4647-9c64-22c55eccf863") : object "kube-system"/"original-pull-secret" not registered Sep 10 22:18:46.447281 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.447259 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:46.447414 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.447357 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs podName:0521c5d3-62eb-4ea7-b2e4-a2da4244d291 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:18.447340073 +0000 UTC m=+65.243555545 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs") pod "network-metrics-daemon-9dbrx" (UID: "0521c5d3-62eb-4ea7-b2e4-a2da4244d291") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 22:18:46.548709 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.548672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:46.555110 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.555076 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrxkn\" (UniqueName: \"kubernetes.io/projected/6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d-kube-api-access-jrxkn\") pod \"network-check-target-cmpvq\" (UID: \"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d\") " pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:46.743412 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.743337 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:18:46.743900 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.743881 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:18:46.745233 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.744337 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:46.746899 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.746377 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 22:18:46.746899 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.746681 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 22:18:46.748117 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.747280 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4rw4f\"" Sep 10 22:18:46.748117 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.747490 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sd8gj\"" Sep 10 22:18:46.753758 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.750447 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:46.753758 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.750612 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:18:46.753758 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.750628 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c7d7d47c8-2k8vr: secret "image-registry-tls" not found Sep 10 22:18:46.753758 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.750688 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls podName:5378dc3b-6b96-428f-abf9-05d5c9ae9e6a nodeName:}" failed. No retries permitted until 2026-09-10 22:18:48.750670478 +0000 UTC m=+35.546885947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls") pod "image-registry-5c7d7d47c8-2k8vr" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a") : secret "image-registry-tls" not found Sep 10 22:18:46.782457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.782331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:46.851936 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.851851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:46.851936 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.851913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:46.852457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.851993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:46.852457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.852044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:46.852457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.852073 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:46.852457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:46.852139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:46.852457 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.852296 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 22:18:46.852457 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.852368 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert podName:ef9fa3f2-f8a4-40e2-aa74-de5824a3b356 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:48.852350199 +0000 UTC m=+35.648565670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rlb5v" (UID: "ef9fa3f2-f8a4-40e2-aa74-de5824a3b356") : secret "networking-console-plugin-cert" not found Sep 10 22:18:46.853114 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853095 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 22:18:46.853205 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853151 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls podName:82c80596-0e41-4e7b-9f6a-c27da394a286 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:48.853133504 +0000 UTC m=+35.649348960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s5v9m" (UID: "82c80596-0e41-4e7b-9f6a-c27da394a286") : secret "samples-operator-tls" not found Sep 10 22:18:46.853266 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853215 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:48.853204638 +0000 UTC m=+35.649420100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : configmap references non-existent config key: service-ca.crt Sep 10 22:18:46.853325 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853280 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:46.853325 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853313 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls podName:d6f97834-3301-409b-a2ea-a6fd959d3068 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:48.853302566 +0000 UTC m=+35.649518023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4pkxw" (UID: "d6f97834-3301-409b-a2ea-a6fd959d3068") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:46.853850 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853625 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 22:18:46.853850 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853664 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:48.853651673 +0000 UTC m=+35.649867133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : secret "router-metrics-certs-default" not found Sep 10 22:18:46.853850 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853712 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:18:46.853850 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:46.853738 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls podName:2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:48.853728684 +0000 UTC m=+35.649944143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls") pod "dns-default-6lz6l" (UID: "2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20") : secret "dns-default-metrics-tls" not found Sep 10 22:18:47.128118 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:47.128082 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmpvq"] Sep 10 22:18:47.132527 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:18:47.132490 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce5ba6d_22f8_4b50_8a91_c4c5d2dd361d.slice/crio-888d8a574791c79416c56d50e30325ad6f96e8038e4a025c70995a30262011da WatchSource:0}: Error finding container 888d8a574791c79416c56d50e30325ad6f96e8038e4a025c70995a30262011da: Status 404 returned error can't find the container with id 888d8a574791c79416c56d50e30325ad6f96e8038e4a025c70995a30262011da Sep 10 22:18:47.987713 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:47.987455 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmpvq" event={"ID":"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d","Type":"ContainerStarted","Data":"888d8a574791c79416c56d50e30325ad6f96e8038e4a025c70995a30262011da"} Sep 10 22:18:48.773907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:48.773114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:48.773907 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.773418 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:18:48.773907 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.773435 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c7d7d47c8-2k8vr: secret "image-registry-tls" not found Sep 10 22:18:48.773907 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.773492 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls podName:5378dc3b-6b96-428f-abf9-05d5c9ae9e6a nodeName:}" failed. No retries permitted until 2026-09-10 22:18:52.77347445 +0000 UTC m=+39.569689922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls") pod "image-registry-5c7d7d47c8-2k8vr" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a") : secret "image-registry-tls" not found Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:48.873620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:48.873734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:48.873755 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:48.873793 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:48.873843 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:48.873874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874022 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874086 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls podName:2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:52.874067815 +0000 UTC m=+39.670283282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls") pod "dns-default-6lz6l" (UID: "2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20") : secret "dns-default-metrics-tls" not found Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874442 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874488 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert podName:ef9fa3f2-f8a4-40e2-aa74-de5824a3b356 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:52.874476203 +0000 UTC m=+39.670691662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rlb5v" (UID: "ef9fa3f2-f8a4-40e2-aa74-de5824a3b356") : secret "networking-console-plugin-cert" not found Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874554 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874592 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls podName:82c80596-0e41-4e7b-9f6a-c27da394a286 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:52.87458107 +0000 UTC m=+39.670796540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s5v9m" (UID: "82c80596-0e41-4e7b-9f6a-c27da394a286") : secret "samples-operator-tls" not found Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874667 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:52.874655628 +0000 UTC m=+39.670871095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : configmap references non-existent config key: service-ca.crt Sep 10 22:18:48.874969 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874729 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:48.875775 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874763 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls podName:d6f97834-3301-409b-a2ea-a6fd959d3068 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:52.874752306 +0000 UTC m=+39.670967775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4pkxw" (UID: "d6f97834-3301-409b-a2ea-a6fd959d3068") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:48.875775 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874837 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 22:18:48.875775 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:48.874872 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:52.874861584 +0000 UTC m=+39.671077047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : secret "router-metrics-certs-default" not found Sep 10 22:18:52.810918 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:52.810875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:18:52.811270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.811022 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:18:52.811270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.811050 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c7d7d47c8-2k8vr: secret "image-registry-tls" not found Sep 10 22:18:52.811270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.811106 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls podName:5378dc3b-6b96-428f-abf9-05d5c9ae9e6a nodeName:}" failed. No retries permitted until 2026-09-10 22:19:00.811091166 +0000 UTC m=+47.607306624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls") pod "image-registry-5c7d7d47c8-2k8vr" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a") : secret "image-registry-tls" not found Sep 10 22:18:52.912039 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:52.911999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:18:52.912039 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:52.912039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:52.912270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:52.912095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:18:52.912270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:52.912128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:18:52.912270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912132 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 22:18:52.912270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:52.912154 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:18:52.912270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912213 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls podName:82c80596-0e41-4e7b-9f6a-c27da394a286 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:00.912191223 +0000 UTC m=+47.708406721 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s5v9m" (UID: "82c80596-0e41-4e7b-9f6a-c27da394a286") : secret "samples-operator-tls" not found Sep 10 22:18:52.912270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912245 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:00.912226962 +0000 UTC m=+47.708442439 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : configmap references non-existent config key: service-ca.crt Sep 10 22:18:52.912270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912249 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:52.912287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912298 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls podName:2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:00.912287637 +0000 UTC m=+47.708503093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls") pod "dns-default-6lz6l" (UID: "2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20") : secret "dns-default-metrics-tls" not found Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912351 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912383 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert podName:ef9fa3f2-f8a4-40e2-aa74-de5824a3b356 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:00.912373331 +0000 UTC m=+47.708588789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rlb5v" (UID: "ef9fa3f2-f8a4-40e2-aa74-de5824a3b356") : secret "networking-console-plugin-cert" not found Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912405 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912428 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912437 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls podName:d6f97834-3301-409b-a2ea-a6fd959d3068 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:00.912426711 +0000 UTC m=+47.708642169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4pkxw" (UID: "d6f97834-3301-409b-a2ea-a6fd959d3068") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:18:52.912637 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:52.912457 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:00.912447608 +0000 UTC m=+47.708663064 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : secret "router-metrics-certs-default" not found Sep 10 22:18:57.018407 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.018158 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" event={"ID":"e9f0338c-f42f-4873-8ebe-7ae3d7541cda","Type":"ContainerStarted","Data":"678d4a1b1d6902ff27c47554ba19d62222c958ce71aa3c906b3a99a6c198ea03"} Sep 10 22:18:57.020780 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.020639 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" event={"ID":"75d82860-197d-4819-9059-222e9c024ddf","Type":"ContainerStarted","Data":"8ddd6f3cf6b7332b2940edf268cbed2ea8c0f9e9e8005a98f259beb1a0cfeaf1"} Sep 10 22:18:57.022576 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.022541 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jv4bp" event={"ID":"099a0642-5254-4571-8da9-e6cce52ff27f","Type":"ContainerStarted","Data":"96032bc9e6362f8897bb4a1189e638d8818b18d6c68198cd5fee352de7438bbd"} Sep 10 22:18:57.024583 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.024561 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmpvq" event={"ID":"6ce5ba6d-22f8-4b50-8a91-c4c5d2dd361d","Type":"ContainerStarted","Data":"60c61d952608279a68b613ddffd53481640fd3d5fd7b66846cfbbd9ed4913238"} Sep 10 22:18:57.024842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.024617 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:18:57.026393 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.026241 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n9pms" event={"ID":"ef748efe-3001-4cdf-b168-2b215ee1cd9a","Type":"ContainerStarted","Data":"07d3da335335dbb71340ec3506e48828f94e1689f3d751ae9a97c3cd6352d1a4"} Sep 10 22:18:57.027992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.027959 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" event={"ID":"367f7843-3bbe-4c67-8a5c-cc7a03f5b123","Type":"ContainerStarted","Data":"12fa365f79cce85581eb84504b8161a94eb3c6ea193d3e5695a6a91bc90f73d6"} Sep 10 22:18:57.029981 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.029961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" event={"ID":"13205e70-75c2-4340-a014-0f3d44963cbd","Type":"ContainerStarted","Data":"4bbb9572d06a72ec1bb3636ef04117081e0c216dd448f0b3777eefa21b79090d"} Sep 10 22:18:57.031853 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.031787 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/0.log" Sep 10 22:18:57.031985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.031874 2577 generic.go:356] "Generic (PLEG): container finished" podID="84a7c959-b185-469d-8a42-30171c664a31" containerID="80cebfc69a0bb439b8d44b9247c265d38156f7b34c668c8a715dc4ebef81550d" exitCode=255 Sep 10 22:18:57.031985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.031980 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-866j5" event={"ID":"84a7c959-b185-469d-8a42-30171c664a31","Type":"ContainerDied","Data":"80cebfc69a0bb439b8d44b9247c265d38156f7b34c668c8a715dc4ebef81550d"} Sep 10 22:18:57.032099 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.032083 2577 scope.go:117] "RemoveContainer" containerID="80cebfc69a0bb439b8d44b9247c265d38156f7b34c668c8a715dc4ebef81550d" Sep 10 22:18:57.033366 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.033297 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" podStartSLOduration=19.881339814 podStartE2EDuration="30.033282446s" totalImagesPullingTime="10.151942632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:57.032420502 +0000 UTC m=+43.828635980" watchObservedRunningTime="2026-09-10 22:18:57.033282446 +0000 UTC m=+43.829497925" Sep 10 22:18:57.037394 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.037372 2577 generic.go:356] "Generic (PLEG): container finished" podID="079e30d5-078c-4982-981d-f4df3a17834b" containerID="d3739a786144b118bd032285b175042d3d11e03b1dd4b1ea2cbb3b4c8868d353" exitCode=0 Sep 10 22:18:57.037506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.037412 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerDied","Data":"d3739a786144b118bd032285b175042d3d11e03b1dd4b1ea2cbb3b4c8868d353"} Sep 10 22:18:57.065016 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.064973 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-q44g9" podStartSLOduration=19.708458597 podStartE2EDuration="30.064959981s" totalImagesPullingTime="10.356501384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:57.064352788 +0000 UTC m=+43.860568266" watchObservedRunningTime="2026-09-10 22:18:57.064959981 +0000 UTC m=+43.861175453" Sep 10 22:18:57.099099 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.097606 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cmpvq" podStartSLOduration=34.91590541 podStartE2EDuration="44.0975872s" totalImagesPullingTime="9.18168179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:57.081295039 +0000 UTC m=+43.877510517" watchObservedRunningTime="2026-09-10 22:18:57.0975872 +0000 UTC m=+43.893802667" Sep 10 22:18:57.115730 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.115682 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n9pms" podStartSLOduration=1.996370706 podStartE2EDuration="12.115651553s" totalImagesPullingTime="10.119280847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:57.098899657 +0000 UTC m=+43.895115136" watchObservedRunningTime="2026-09-10 22:18:57.115651553 +0000 UTC m=+43.911867031" Sep 10 22:18:57.145359 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.145231 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" podStartSLOduration=22.039488377 podStartE2EDuration="32.145212879s" totalImagesPullingTime="10.105724502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:57.116307657 +0000 UTC m=+43.912523138" watchObservedRunningTime="2026-09-10 22:18:57.145212879 +0000 UTC m=+43.941428357" Sep 10 22:18:57.146756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.146710 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-jv4bp" podStartSLOduration=22.995798123 podStartE2EDuration="33.146696533s" totalImagesPullingTime="10.15089841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:57.14474601 +0000 UTC m=+43.940961492" watchObservedRunningTime="2026-09-10 22:18:57.146696533 +0000 UTC m=+43.942912015" Sep 10 22:18:57.166788 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:57.166749 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-frh8v" podStartSLOduration=23.087210421 podStartE2EDuration="33.166736234s" totalImagesPullingTime="10.079525813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:57.162445792 +0000 UTC m=+43.958661271" watchObservedRunningTime="2026-09-10 22:18:57.166736234 +0000 UTC m=+43.962951711" Sep 10 22:18:58.043120 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.042901 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:18:58.044197 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.044173 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/0.log" Sep 10 22:18:58.044347 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.044220 2577 generic.go:356] "Generic (PLEG): container finished" podID="84a7c959-b185-469d-8a42-30171c664a31" containerID="5b40ba1ebaa8d3420ee4dfa21d5d56d15f0520590919c1444aba9cf4d778892b" exitCode=255 Sep 10 22:18:58.044347 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.044322 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-866j5" event={"ID":"84a7c959-b185-469d-8a42-30171c664a31","Type":"ContainerDied","Data":"5b40ba1ebaa8d3420ee4dfa21d5d56d15f0520590919c1444aba9cf4d778892b"} Sep 10 22:18:58.044463 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.044374 2577 scope.go:117] "RemoveContainer" containerID="80cebfc69a0bb439b8d44b9247c265d38156f7b34c668c8a715dc4ebef81550d" Sep 10 22:18:58.044693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.044674 2577 scope.go:117] "RemoveContainer" containerID="5b40ba1ebaa8d3420ee4dfa21d5d56d15f0520590919c1444aba9cf4d778892b" Sep 10 22:18:58.044976 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.044951 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-866j5_openshift-console-operator(84a7c959-b185-469d-8a42-30171c664a31)\"" pod="openshift-console-operator/console-operator-696884fb9c-866j5" podUID="84a7c959-b185-469d-8a42-30171c664a31" Sep 10 22:18:58.050440 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.050413 2577 generic.go:356] "Generic (PLEG): container finished" podID="079e30d5-078c-4982-981d-f4df3a17834b" containerID="0f8a25360caa325cfa2bab2306d53667bb22e9514f4f3691fc7eca821ddfab52" exitCode=0 Sep 10 22:18:58.051139 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.051118 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerDied","Data":"0f8a25360caa325cfa2bab2306d53667bb22e9514f4f3691fc7eca821ddfab52"} Sep 10 22:18:58.105374 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.105324 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hd4lz"] Sep 10 22:18:58.139373 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.139350 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hd4lz"] Sep 10 22:18:58.139503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.139482 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.141704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.141686 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 22:18:58.141784 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.141692 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-27trn\"" Sep 10 22:18:58.141978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.141964 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 22:18:58.173322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.173301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.173426 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.173341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/52cf965f-2db0-45c4-b930-5cb0908b6499-data-volume\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.173426 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.173374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqw5f\" (UniqueName: \"kubernetes.io/projected/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-api-access-vqw5f\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.173426 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.173398 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/52cf965f-2db0-45c4-b930-5cb0908b6499-crio-socket\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.173557 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.173509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.274796 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.274766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqw5f\" (UniqueName: \"kubernetes.io/projected/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-api-access-vqw5f\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.274927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.274802 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/52cf965f-2db0-45c4-b930-5cb0908b6499-crio-socket\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.274927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.274917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.275041 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.275013 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:18:58.275041 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.275025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.275134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.275069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/52cf965f-2db0-45c4-b930-5cb0908b6499-data-volume\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.275134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.275106 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/52cf965f-2db0-45c4-b930-5cb0908b6499-crio-socket\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.275214 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.275146 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:58.775120689 +0000 UTC m=+45.571336145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : secret "insights-runtime-extractor-tls" not found Sep 10 22:18:58.275214 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.275148 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:18:58.275214 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.275207 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:58.775189509 +0000 UTC m=+45.571404968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : configmap "kube-rbac-proxy" not found Sep 10 22:18:58.275357 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.275341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/52cf965f-2db0-45c4-b930-5cb0908b6499-data-volume\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.288152 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.288130 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqw5f\" (UniqueName: \"kubernetes.io/projected/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-api-access-vqw5f\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.779581 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.779550 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:58.779841 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.779670 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:18:58.779841 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.779757 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:18:58.779841 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.779770 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:59.779751103 +0000 UTC m=+46.575966575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : configmap "kube-rbac-proxy" not found Sep 10 22:18:58.779841 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:58.779797 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:18:59.779787405 +0000 UTC m=+46.576002866 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : secret "insights-runtime-extractor-tls" not found Sep 10 22:18:58.779841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:58.779669 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:59.059987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:59.059921 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:18:59.060849 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:59.060244 2577 scope.go:117] "RemoveContainer" containerID="5b40ba1ebaa8d3420ee4dfa21d5d56d15f0520590919c1444aba9cf4d778892b" Sep 10 22:18:59.060849 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:59.060473 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-866j5_openshift-console-operator(84a7c959-b185-469d-8a42-30171c664a31)\"" pod="openshift-console-operator/console-operator-696884fb9c-866j5" podUID="84a7c959-b185-469d-8a42-30171c664a31" Sep 10 22:18:59.063409 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:59.063384 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j454d" event={"ID":"079e30d5-078c-4982-981d-f4df3a17834b","Type":"ContainerStarted","Data":"9324486234e6759e885f682dbe0fa261a164a4775c3af149223a00108c46c2a7"} Sep 10 22:18:59.093811 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:59.093768 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j454d" podStartSLOduration=12.332388194 podStartE2EDuration="46.093754062s" totalImagesPullingTime="33.761365868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:18:59.092052721 +0000 UTC m=+45.888268198" watchObservedRunningTime="2026-09-10 22:18:59.093754062 +0000 UTC m=+45.889969553" Sep 10 22:18:59.789922 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:59.789887 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:59.790078 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:18:59.789985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:18:59.790078 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:59.790008 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:18:59.790078 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:59.790065 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:01.790050752 +0000 UTC m=+48.586266206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : configmap "kube-rbac-proxy" not found Sep 10 22:18:59.790183 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:59.790093 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:18:59.790183 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:18:59.790136 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:01.79012117 +0000 UTC m=+48.586336644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : secret "insights-runtime-extractor-tls" not found Sep 10 22:19:00.262654 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.262583 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nj85g_63da62fc-f1bc-49e9-b2ff-0a39cf81331f/dns-node-resolver/0.log" Sep 10 22:19:00.898916 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.898876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:19:00.899072 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.899035 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 22:19:00.899072 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.899052 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c7d7d47c8-2k8vr: secret "image-registry-tls" not found Sep 10 22:19:00.899142 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.899109 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls podName:5378dc3b-6b96-428f-abf9-05d5c9ae9e6a nodeName:}" failed. No retries permitted until 2026-09-10 22:19:16.89909402 +0000 UTC m=+63.695309501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls") pod "image-registry-5c7d7d47c8-2k8vr" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a") : secret "image-registry-tls" not found Sep 10 22:19:00.999409 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.999379 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:19:00.999409 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.999411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:00.999595 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.999439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:19:00.999595 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.999470 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:00.999595 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.999499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:19:00.999595 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999536 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 10 22:19:00.999595 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999537 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 10 22:19:00.999595 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999582 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:16.999542307 +0000 UTC m=+63.795757780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : configmap references non-existent config key: service-ca.crt Sep 10 22:19:00.999595 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999594 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999620 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls podName:d6f97834-3301-409b-a2ea-a6fd959d3068 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:16.999598273 +0000 UTC m=+63.795813743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-4pkxw" (UID: "d6f97834-3301-409b-a2ea-a6fd959d3068") : secret "cluster-monitoring-operator-tls" not found Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999626 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999649 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls podName:82c80596-0e41-4e7b-9f6a-c27da394a286 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:16.999631046 +0000 UTC m=+63.795846503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-s5v9m" (UID: "82c80596-0e41-4e7b-9f6a-c27da394a286") : secret "samples-operator-tls" not found Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999666 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls podName:2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:16.999660083 +0000 UTC m=+63.795875539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls") pod "dns-default-6lz6l" (UID: "2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20") : secret "dns-default-metrics-tls" not found Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999700 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs podName:9dfbb79d-2457-4d91-8bd0-fe718a39e9f4 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:16.999683454 +0000 UTC m=+63.795898922 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs") pod "router-default-6cf6d56fcd-xqxl8" (UID: "9dfbb79d-2457-4d91-8bd0-fe718a39e9f4") : secret "router-metrics-certs-default" not found Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:00.999727 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999885 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 22:19:00.999937 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:00.999935 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert podName:ef9fa3f2-f8a4-40e2-aa74-de5824a3b356 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:16.999918925 +0000 UTC m=+63.796134394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-rlb5v" (UID: "ef9fa3f2-f8a4-40e2-aa74-de5824a3b356") : secret "networking-console-plugin-cert" not found Sep 10 22:19:01.064482 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:01.064461 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rstb7_57467133-81f3-434e-ae9f-1f07cd84ceba/node-ca/0.log" Sep 10 22:19:01.805622 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:01.805587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:01.806044 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:01.805684 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:01.806044 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:01.805709 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:19:01.806044 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:01.805762 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:05.805748173 +0000 UTC m=+52.601963665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : configmap "kube-rbac-proxy" not found Sep 10 22:19:01.806044 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:01.805768 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:19:01.806044 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:01.805832 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:05.805802598 +0000 UTC m=+52.602018052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : secret "insights-runtime-extractor-tls" not found Sep 10 22:19:01.863245 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:01.863214 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n9pms_ef748efe-3001-4cdf-b168-2b215ee1cd9a/serve-healthcheck-canary/0.log" Sep 10 22:19:05.436872 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:05.436789 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:19:05.436872 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:05.436890 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:19:05.437301 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:05.437236 2577 scope.go:117] "RemoveContainer" containerID="5b40ba1ebaa8d3420ee4dfa21d5d56d15f0520590919c1444aba9cf4d778892b" Sep 10 22:19:05.437410 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:05.437390 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-866j5_openshift-console-operator(84a7c959-b185-469d-8a42-30171c664a31)\"" pod="openshift-console-operator/console-operator-696884fb9c-866j5" podUID="84a7c959-b185-469d-8a42-30171c664a31" Sep 10 22:19:05.844962 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:05.844931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:05.845186 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:05.845006 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:05.845186 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:05.845076 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:19:05.845186 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:05.845084 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 22:19:05.845186 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:05.845138 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:13.845124845 +0000 UTC m=+60.641340314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : configmap "kube-rbac-proxy" not found Sep 10 22:19:05.845186 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:05.845150 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:13.845144627 +0000 UTC m=+60.641360085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : secret "insights-runtime-extractor-tls" not found Sep 10 22:19:13.908588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:13.908547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:13.909015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:13.908683 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:13.909015 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:13.908856 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 22:19:13.909015 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:13.908922 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls podName:52cf965f-2db0-45c4-b930-5cb0908b6499 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:29.908906041 +0000 UTC m=+76.705121497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hd4lz" (UID: "52cf965f-2db0-45c4-b930-5cb0908b6499") : secret "insights-runtime-extractor-tls" not found Sep 10 22:19:13.909135 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:13.909100 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/52cf965f-2db0-45c4-b930-5cb0908b6499-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:13.982128 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:13.982104 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-slc2q" Sep 10 22:19:16.933674 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:16.933636 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:19:16.936238 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:16.936217 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"image-registry-5c7d7d47c8-2k8vr\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:19:17.034370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.034339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:19:17.034370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.034374 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:17.034594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.034397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:19:17.034594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.034418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:17.034594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.034435 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:19:17.034748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.034695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:19:17.035138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.035111 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-service-ca-bundle\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:17.036870 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.036850 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20-metrics-tls\") pod \"dns-default-6lz6l\" (UID: \"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20\") " pod="openshift-dns/dns-default-6lz6l" Sep 10 22:19:17.037415 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.037393 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6f97834-3301-409b-a2ea-a6fd959d3068-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-4pkxw\" (UID: \"d6f97834-3301-409b-a2ea-a6fd959d3068\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:19:17.037496 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.037475 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9dfbb79d-2457-4d91-8bd0-fe718a39e9f4-metrics-certs\") pod \"router-default-6cf6d56fcd-xqxl8\" (UID: \"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4\") " pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:17.037632 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.037612 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ef9fa3f2-f8a4-40e2-aa74-de5824a3b356-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-rlb5v\" (UID: \"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:19:17.037670 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.037612 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/82c80596-0e41-4e7b-9f6a-c27da394a286-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-s5v9m\" (UID: \"82c80596-0e41-4e7b-9f6a-c27da394a286\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:19:17.170465 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.170442 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6vhns\"" Sep 10 22:19:17.179157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.179143 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:19:17.193717 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.193672 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pflv9\"" Sep 10 22:19:17.202359 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.202345 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" Sep 10 22:19:17.214702 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.214688 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zm8fb\"" Sep 10 22:19:17.223339 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.223312 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" Sep 10 22:19:17.225095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.225076 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bjgt9\"" Sep 10 22:19:17.233908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.233882 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6lz6l" Sep 10 22:19:17.245629 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.244928 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sn269\"" Sep 10 22:19:17.252696 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.252673 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:17.253240 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.253223 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-7529d\"" Sep 10 22:19:17.262187 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.262125 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" Sep 10 22:19:17.483848 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.483748 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c7d7d47c8-2k8vr"] Sep 10 22:19:17.486548 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:17.486526 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5378dc3b_6b96_428f_abf9_05d5c9ae9e6a.slice/crio-f8f4de25e5163601ce124a04cc6c31dae9dc00ea033fcbea4a88bce6863be009 WatchSource:0}: Error finding container f8f4de25e5163601ce124a04cc6c31dae9dc00ea033fcbea4a88bce6863be009: Status 404 returned error can't find the container with id f8f4de25e5163601ce124a04cc6c31dae9dc00ea033fcbea4a88bce6863be009 Sep 10 22:19:17.514683 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.514666 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m"] Sep 10 22:19:17.560690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.560667 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw"] Sep 10 22:19:17.563404 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:17.563384 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f97834_3301_409b_a2ea_a6fd959d3068.slice/crio-bd72b69e8b5a1ab218c8c2191af343264fd61d509c95b1f1391301072a8c6164 WatchSource:0}: Error finding container bd72b69e8b5a1ab218c8c2191af343264fd61d509c95b1f1391301072a8c6164: Status 404 returned error can't find the container with id bd72b69e8b5a1ab218c8c2191af343264fd61d509c95b1f1391301072a8c6164 Sep 10 22:19:17.576236 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.576208 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6cf6d56fcd-xqxl8"] Sep 10 22:19:17.578385 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:17.578364 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dfbb79d_2457_4d91_8bd0_fe718a39e9f4.slice/crio-d9748bf3799044b45d10685d9060e9cfb8b65bc9a55396571e11d824fded9ee6 WatchSource:0}: Error finding container d9748bf3799044b45d10685d9060e9cfb8b65bc9a55396571e11d824fded9ee6: Status 404 returned error can't find the container with id d9748bf3799044b45d10685d9060e9cfb8b65bc9a55396571e11d824fded9ee6 Sep 10 22:19:17.605949 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.605923 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6lz6l"] Sep 10 22:19:17.609081 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:17.609056 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f59c5a1_f8f2_4ee2_96de_ea377a4d7c20.slice/crio-7574154fd345189d0cd219c039278c9dcb9a840dcdf75e963a1da0bc9ed3860f WatchSource:0}: Error finding container 7574154fd345189d0cd219c039278c9dcb9a840dcdf75e963a1da0bc9ed3860f: Status 404 returned error can't find the container with id 7574154fd345189d0cd219c039278c9dcb9a840dcdf75e963a1da0bc9ed3860f Sep 10 22:19:17.623314 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.623295 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v"] Sep 10 22:19:17.744119 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:17.744045 2577 scope.go:117] "RemoveContainer" containerID="5b40ba1ebaa8d3420ee4dfa21d5d56d15f0520590919c1444aba9cf4d778892b" Sep 10 22:19:18.127126 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.127089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" event={"ID":"d6f97834-3301-409b-a2ea-a6fd959d3068","Type":"ContainerStarted","Data":"bd72b69e8b5a1ab218c8c2191af343264fd61d509c95b1f1391301072a8c6164"} Sep 10 22:19:18.129413 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.129005 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" event={"ID":"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a","Type":"ContainerStarted","Data":"c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8"} Sep 10 22:19:18.129413 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.129049 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" event={"ID":"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a","Type":"ContainerStarted","Data":"f8f4de25e5163601ce124a04cc6c31dae9dc00ea033fcbea4a88bce6863be009"} Sep 10 22:19:18.129413 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.129181 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:19:18.131308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.131247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6lz6l" event={"ID":"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20","Type":"ContainerStarted","Data":"7574154fd345189d0cd219c039278c9dcb9a840dcdf75e963a1da0bc9ed3860f"} Sep 10 22:19:18.132972 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.132942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" event={"ID":"82c80596-0e41-4e7b-9f6a-c27da394a286","Type":"ContainerStarted","Data":"5c78d159e8a3cc6ea2b213259a2f585e38664922dbc7b21b45db6234b09c6730"} Sep 10 22:19:18.134575 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.134553 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" event={"ID":"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356","Type":"ContainerStarted","Data":"f2d15cfca62c1592f059b29cca6c965d568507e8bba9552f7ded6f2c5f12d72a"} Sep 10 22:19:18.137387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.137363 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" event={"ID":"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4","Type":"ContainerStarted","Data":"2f64c596bbaa1070116b7882bec3b8311b268f4597c6ee6b28624d5e760839c4"} Sep 10 22:19:18.137479 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.137404 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" event={"ID":"9dfbb79d-2457-4d91-8bd0-fe718a39e9f4","Type":"ContainerStarted","Data":"d9748bf3799044b45d10685d9060e9cfb8b65bc9a55396571e11d824fded9ee6"} Sep 10 22:19:18.141441 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.140901 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:19:18.141441 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.140944 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-866j5" event={"ID":"84a7c959-b185-469d-8a42-30171c664a31","Type":"ContainerStarted","Data":"eb2a8c151402548130c2b47efafde2a2c775e327c2947f95ed89b8db58b6516c"} Sep 10 22:19:18.141441 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.141402 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:19:18.150342 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.150299 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" podStartSLOduration=64.150285488 podStartE2EDuration="1m4.150285488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:18.149197401 +0000 UTC m=+64.945412879" watchObservedRunningTime="2026-09-10 22:19:18.150285488 +0000 UTC m=+64.946500964" Sep 10 22:19:18.165750 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.165204 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-866j5" podStartSLOduration=44.01563522 podStartE2EDuration="54.165185449s" totalImagesPullingTime="10.149550229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:18.164619768 +0000 UTC m=+64.960835246" watchObservedRunningTime="2026-09-10 22:19:18.165185449 +0000 UTC m=+64.961400924" Sep 10 22:19:18.181290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.181229 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" podStartSLOduration=54.181217669 podStartE2EDuration="54.181217669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:18.180492939 +0000 UTC m=+64.976708418" watchObservedRunningTime="2026-09-10 22:19:18.181217669 +0000 UTC m=+64.977433145" Sep 10 22:19:18.253520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.253459 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:18.256278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.256097 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:18.449542 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.449461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:19:18.449876 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.449852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:19:18.451726 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.451703 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 22:19:18.452063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.452046 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 22:19:18.480271 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.480239 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d0d3d831-2afd-4647-9c64-22c55eccf863-original-pull-secret\") pod \"global-pull-secret-syncer-g7tpk\" (UID: \"d0d3d831-2afd-4647-9c64-22c55eccf863\") " pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:19:18.480271 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.480255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0521c5d3-62eb-4ea7-b2e4-a2da4244d291-metrics-certs\") pod \"network-metrics-daemon-9dbrx\" (UID: \"0521c5d3-62eb-4ea7-b2e4-a2da4244d291\") " pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:19:18.563342 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.563314 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g7tpk" Sep 10 22:19:18.577055 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.577004 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4rw4f\"" Sep 10 22:19:18.585785 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.585747 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9dbrx" Sep 10 22:19:18.887073 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.886282 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g7tpk"] Sep 10 22:19:18.925948 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:18.925868 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9dbrx"] Sep 10 22:19:19.129596 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:19.129534 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-866j5" Sep 10 22:19:19.149412 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:19.149349 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:19.150927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:19.150902 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6cf6d56fcd-xqxl8" Sep 10 22:19:20.410551 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:20.410519 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d3d831_2afd_4647_9c64_22c55eccf863.slice/crio-e0f519baf77f35aeb3e811ae3a707c814597350aacf39ba5e60a938ebc51f1fa WatchSource:0}: Error finding container e0f519baf77f35aeb3e811ae3a707c814597350aacf39ba5e60a938ebc51f1fa: Status 404 returned error can't find the container with id e0f519baf77f35aeb3e811ae3a707c814597350aacf39ba5e60a938ebc51f1fa Sep 10 22:19:20.411478 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:20.411272 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0521c5d3_62eb_4ea7_b2e4_a2da4244d291.slice/crio-7bd4fb215fccaad71ff3079a1a8277ff471cbcfeac92a238c9692cf6f3663f26 WatchSource:0}: Error finding container 7bd4fb215fccaad71ff3079a1a8277ff471cbcfeac92a238c9692cf6f3663f26: Status 404 returned error can't find the container with id 7bd4fb215fccaad71ff3079a1a8277ff471cbcfeac92a238c9692cf6f3663f26 Sep 10 22:19:21.154934 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:21.154898 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g7tpk" event={"ID":"d0d3d831-2afd-4647-9c64-22c55eccf863","Type":"ContainerStarted","Data":"e0f519baf77f35aeb3e811ae3a707c814597350aacf39ba5e60a938ebc51f1fa"} Sep 10 22:19:21.155996 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:21.155966 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dbrx" event={"ID":"0521c5d3-62eb-4ea7-b2e4-a2da4244d291","Type":"ContainerStarted","Data":"7bd4fb215fccaad71ff3079a1a8277ff471cbcfeac92a238c9692cf6f3663f26"} Sep 10 22:19:22.165284 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:22.165248 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" event={"ID":"ef9fa3f2-f8a4-40e2-aa74-de5824a3b356","Type":"ContainerStarted","Data":"b521cbbb624b2e4958e15dc2dfc1ef81986afb108d009f0c45aa33cb224670cb"} Sep 10 22:19:22.168631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:22.168544 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" event={"ID":"d6f97834-3301-409b-a2ea-a6fd959d3068","Type":"ContainerStarted","Data":"f615a487fdc4042085ebccaf6ad18e76a855cd2e918675d3c587553ad3b90d7a"} Sep 10 22:19:22.171310 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:22.171276 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6lz6l" event={"ID":"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20","Type":"ContainerStarted","Data":"e39917b9f416d319753090ab7feaee18c7521a5e342f6b6189b4408f5f401a19"} Sep 10 22:19:22.173770 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:22.173745 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" event={"ID":"82c80596-0e41-4e7b-9f6a-c27da394a286","Type":"ContainerStarted","Data":"af722436afb9fa43d536b24f6a862eafe40c29ee2952cb418a33bbc5adfbe0cf"} Sep 10 22:19:22.198219 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:22.198077 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-rlb5v" podStartSLOduration=45.861652921 podStartE2EDuration="50.198064313s" totalImagesPullingTime="4.336411392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:22.196726441 +0000 UTC m=+68.992941921" watchObservedRunningTime="2026-09-10 22:19:22.198064313 +0000 UTC m=+68.994279804" Sep 10 22:19:22.213781 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:22.213736 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-4pkxw" podStartSLOduration=53.81138106 podStartE2EDuration="58.213719186s" totalImagesPullingTime="4.402338126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:22.213498173 +0000 UTC m=+69.009713649" watchObservedRunningTime="2026-09-10 22:19:22.213719186 +0000 UTC m=+69.009934687" Sep 10 22:19:23.181971 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.181929 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6lz6l" event={"ID":"2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20","Type":"ContainerStarted","Data":"a01381a4d62bbab2f4e502c85567814a4b0966afcce185a15ae4fcf4a301cf24"} Sep 10 22:19:23.182508 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.182042 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6lz6l" Sep 10 22:19:23.184441 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.184103 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" event={"ID":"82c80596-0e41-4e7b-9f6a-c27da394a286","Type":"ContainerStarted","Data":"2e9d32f88f7808dd351a4674dc5c59733e4438d1548663c1cc500d98d45b78e6"} Sep 10 22:19:23.186554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.186516 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dbrx" event={"ID":"0521c5d3-62eb-4ea7-b2e4-a2da4244d291","Type":"ContainerStarted","Data":"b776ee9614fa74f0fba04c0a3b5ff995e5bcc32898f500512c8d96db4862031a"} Sep 10 22:19:23.186662 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.186573 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9dbrx" event={"ID":"0521c5d3-62eb-4ea7-b2e4-a2da4244d291","Type":"ContainerStarted","Data":"ba2bc0d8066505d9950c8afdc57010fdff768783e0d5d32c8bbf09037c52f822"} Sep 10 22:19:23.211721 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.211667 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6lz6l" podStartSLOduration=33.854639456 podStartE2EDuration="38.211649794s" totalImagesPullingTime="4.357010338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:23.210539503 +0000 UTC m=+70.006755004" watchObservedRunningTime="2026-09-10 22:19:23.211649794 +0000 UTC m=+70.007865300" Sep 10 22:19:23.240186 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.240139 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9dbrx" podStartSLOduration=68.34703603 podStartE2EDuration="1m10.240124957s" totalImagesPullingTime="1.893088927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:23.238413242 +0000 UTC m=+70.034628742" watchObservedRunningTime="2026-09-10 22:19:23.240124957 +0000 UTC m=+70.036340435" Sep 10 22:19:23.262866 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:23.262792 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-s5v9m" podStartSLOduration=54.870142745 podStartE2EDuration="59.262778493s" totalImagesPullingTime="4.392635748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:23.261262398 +0000 UTC m=+70.057477902" watchObservedRunningTime="2026-09-10 22:19:23.262778493 +0000 UTC m=+70.058993999" Sep 10 22:19:25.196692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:25.196654 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g7tpk" event={"ID":"d0d3d831-2afd-4647-9c64-22c55eccf863","Type":"ContainerStarted","Data":"2717c86bdf4946f1c866a31b866802f090a455444f8a22554b87365da83bdb90"} Sep 10 22:19:25.214043 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:25.213992 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-g7tpk" podStartSLOduration=67.883662916 podStartE2EDuration="1m12.213978433s" totalImagesPullingTime="4.330315517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:25.21296297 +0000 UTC m=+72.009178448" watchObservedRunningTime="2026-09-10 22:19:25.213978433 +0000 UTC m=+72.010193910" Sep 10 22:19:28.054302 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:28.054269 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cmpvq" Sep 10 22:19:29.948951 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:29.948919 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:29.951151 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:29.951124 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/52cf965f-2db0-45c4-b930-5cb0908b6499-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hd4lz\" (UID: \"52cf965f-2db0-45c4-b930-5cb0908b6499\") " pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:29.957378 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:29.957359 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-27trn\"" Sep 10 22:19:29.965178 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:29.965159 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hd4lz" Sep 10 22:19:30.234589 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:30.234518 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hd4lz"] Sep 10 22:19:30.237774 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:30.237747 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52cf965f_2db0_45c4_b930_5cb0908b6499.slice/crio-f01eccd92a91e7685fd5a53d2bbf975c30a8a59833088adb540e85429244395a WatchSource:0}: Error finding container f01eccd92a91e7685fd5a53d2bbf975c30a8a59833088adb540e85429244395a: Status 404 returned error can't find the container with id f01eccd92a91e7685fd5a53d2bbf975c30a8a59833088adb540e85429244395a Sep 10 22:19:31.220443 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:31.220362 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hd4lz" event={"ID":"52cf965f-2db0-45c4-b930-5cb0908b6499","Type":"ContainerStarted","Data":"fef933c4c1aac2d7cd8e0547270d465ec27c4ebab6aadd7dc0be6e31b584fd87"} Sep 10 22:19:31.220443 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:31.220425 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hd4lz" event={"ID":"52cf965f-2db0-45c4-b930-5cb0908b6499","Type":"ContainerStarted","Data":"685ce6e3d3e3f31b7ab6166b742517e777c89dd513f7c283b0e7b77a5a30eb4e"} Sep 10 22:19:31.220801 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:31.220453 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hd4lz" event={"ID":"52cf965f-2db0-45c4-b930-5cb0908b6499","Type":"ContainerStarted","Data":"f01eccd92a91e7685fd5a53d2bbf975c30a8a59833088adb540e85429244395a"} Sep 10 22:19:33.194090 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:33.194055 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6lz6l" Sep 10 22:19:33.230765 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:33.230736 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hd4lz" event={"ID":"52cf965f-2db0-45c4-b930-5cb0908b6499","Type":"ContainerStarted","Data":"a5d721902d688d7e2d6608ae0c35077cf4aff9e48fb0c888a6a350a90cbd50c3"} Sep 10 22:19:33.248790 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:33.248744 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hd4lz" podStartSLOduration=33.322990838 podStartE2EDuration="35.248728813s" totalImagesPullingTime="1.925737975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:18:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:33.247918502 +0000 UTC m=+80.044133980" watchObservedRunningTime="2026-09-10 22:19:33.248728813 +0000 UTC m=+80.044944291" Sep 10 22:19:35.143779 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.143659 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8"] Sep 10 22:19:35.146031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.146015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.148568 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.148542 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 10 22:19:35.148568 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.148565 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7n5gr\"" Sep 10 22:19:35.149171 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.149155 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 22:19:35.149241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.149189 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 10 22:19:35.162390 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.162368 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8"] Sep 10 22:19:35.196191 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.196166 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7js62"] Sep 10 22:19:35.198565 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.198548 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.200825 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.200798 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 22:19:35.200993 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.200967 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 22:19:35.201274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.201251 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tmkzm\"" Sep 10 22:19:35.204981 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.204963 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 22:19:35.290002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.289974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ba225a-2b83-4110-be11-59a10739dac6-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.290002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290007 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8v5r\" (UniqueName: \"kubernetes.io/projected/a1ba225a-2b83-4110-be11-59a10739dac6-kube-api-access-n8v5r\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.290200 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290027 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-sys\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290200 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290042 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-tls\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290200 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290061 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290200 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-wtmp\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290200 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1ba225a-2b83-4110-be11-59a10739dac6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.290370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290216 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-root\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290232 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-textfile\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290252 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrm6v\" (UniqueName: \"kubernetes.io/projected/4dcfaacc-843c-4e92-9570-bd0fbde357d3-kube-api-access-vrm6v\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290324 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcfaacc-843c-4e92-9570-bd0fbde357d3-metrics-client-ca\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.290370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.290360 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ba225a-2b83-4110-be11-59a10739dac6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.391651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ba225a-2b83-4110-be11-59a10739dac6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.391651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ba225a-2b83-4110-be11-59a10739dac6-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8v5r\" (UniqueName: \"kubernetes.io/projected/a1ba225a-2b83-4110-be11-59a10739dac6-kube-api-access-n8v5r\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-sys\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391752 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-tls\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391782 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391849 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-wtmp\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391880 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-sys\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1ba225a-2b83-4110-be11-59a10739dac6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391949 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-root\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.391975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.391977 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-textfile\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrm6v\" (UniqueName: \"kubernetes.io/projected/4dcfaacc-843c-4e92-9570-bd0fbde357d3-kube-api-access-vrm6v\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcfaacc-843c-4e92-9570-bd0fbde357d3-metrics-client-ca\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392211 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-wtmp\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392840 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392503 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-textfile\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392840 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392538 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392840 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4dcfaacc-843c-4e92-9570-bd0fbde357d3-root\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.392840 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392683 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a1ba225a-2b83-4110-be11-59a10739dac6-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.393063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.392929 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcfaacc-843c-4e92-9570-bd0fbde357d3-metrics-client-ca\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.394457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.394397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-tls\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.394559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.394514 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a1ba225a-2b83-4110-be11-59a10739dac6-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.394559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.394521 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a1ba225a-2b83-4110-be11-59a10739dac6-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.394878 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.394861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcfaacc-843c-4e92-9570-bd0fbde357d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.400835 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.400799 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8v5r\" (UniqueName: \"kubernetes.io/projected/a1ba225a-2b83-4110-be11-59a10739dac6-kube-api-access-n8v5r\") pod \"openshift-state-metrics-67dcf8c979-47cr8\" (UID: \"a1ba225a-2b83-4110-be11-59a10739dac6\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.401093 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.401076 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrm6v\" (UniqueName: \"kubernetes.io/projected/4dcfaacc-843c-4e92-9570-bd0fbde357d3-kube-api-access-vrm6v\") pod \"node-exporter-7js62\" (UID: \"4dcfaacc-843c-4e92-9570-bd0fbde357d3\") " pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.455433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.455394 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" Sep 10 22:19:35.507116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.507065 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7js62" Sep 10 22:19:35.521171 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:35.521138 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dcfaacc_843c_4e92_9570_bd0fbde357d3.slice/crio-e2865ed09953d8a129bfbc716ded79118513ef004032ef431ca912b1bddefb10 WatchSource:0}: Error finding container e2865ed09953d8a129bfbc716ded79118513ef004032ef431ca912b1bddefb10: Status 404 returned error can't find the container with id e2865ed09953d8a129bfbc716ded79118513ef004032ef431ca912b1bddefb10 Sep 10 22:19:35.726985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:35.726910 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8"] Sep 10 22:19:35.729755 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:35.729729 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ba225a_2b83_4110_be11_59a10739dac6.slice/crio-7a698b1d4b39f39c76ab13a57ff6eefec849e3e60967769629d0519b7bf51ac6 WatchSource:0}: Error finding container 7a698b1d4b39f39c76ab13a57ff6eefec849e3e60967769629d0519b7bf51ac6: Status 404 returned error can't find the container with id 7a698b1d4b39f39c76ab13a57ff6eefec849e3e60967769629d0519b7bf51ac6 Sep 10 22:19:36.226102 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.226072 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:19:36.228706 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.228689 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.230944 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.230916 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 22:19:36.230944 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.230927 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 22:19:36.230944 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.230927 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 22:19:36.231164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.230976 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 22:19:36.231164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.230927 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 22:19:36.231325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.231308 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 22:19:36.231403 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.231386 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 22:19:36.231512 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.231496 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 22:19:36.231622 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.231605 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-d22jk\"" Sep 10 22:19:36.232078 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.232063 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 22:19:36.241793 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.241771 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7js62" event={"ID":"4dcfaacc-843c-4e92-9570-bd0fbde357d3","Type":"ContainerStarted","Data":"e2865ed09953d8a129bfbc716ded79118513ef004032ef431ca912b1bddefb10"} Sep 10 22:19:36.243591 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.243572 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" event={"ID":"a1ba225a-2b83-4110-be11-59a10739dac6","Type":"ContainerStarted","Data":"92c551829dd7aac928eb14129009c872051cd915e8f817c25e6575a083b20d9d"} Sep 10 22:19:36.243674 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.243611 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" event={"ID":"a1ba225a-2b83-4110-be11-59a10739dac6","Type":"ContainerStarted","Data":"ba6be29be56de43f6d727c2954790c563e97352eeb360b41c9f2332cce076c1f"} Sep 10 22:19:36.243674 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.243629 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" event={"ID":"a1ba225a-2b83-4110-be11-59a10739dac6","Type":"ContainerStarted","Data":"7a698b1d4b39f39c76ab13a57ff6eefec849e3e60967769629d0519b7bf51ac6"} Sep 10 22:19:36.249418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.249398 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:19:36.400804 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.400765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.400978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.400809 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.400978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.400880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.400978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.400914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.400962 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnnb9\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-kube-api-access-rnnb9\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401040 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401070 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-web-config\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401102 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401127 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401181 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-out\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401208 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.401456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.401239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnnb9\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-kube-api-access-rnnb9\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502183 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502209 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502227 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-web-config\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502288 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502330 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-out\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502430 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.502842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502533 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.503000 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.502923 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.504018 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:36.503719 2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 10 22:19:36.504018 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.503725 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.504018 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.503725 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.504018 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:36.503792 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls podName:1c387fb1-35f5-423c-91fe-534c62ee0d3c nodeName:}" failed. No retries permitted until 2026-09-10 22:19:37.003772361 +0000 UTC m=+83.799987824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c") : secret "alertmanager-main-tls" not found Sep 10 22:19:36.507683 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.507622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-out\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.508023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.507974 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-web-config\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.508023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.508012 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.508496 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.508468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.508865 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.508806 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.509693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.509544 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.509693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.509642 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.509854 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.509736 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:36.512056 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:36.512030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnnb9\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-kube-api-access-rnnb9\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:37.006385 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.006348 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:37.006532 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:37.006513 2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Sep 10 22:19:37.006597 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:37.006573 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls podName:1c387fb1-35f5-423c-91fe-534c62ee0d3c nodeName:}" failed. No retries permitted until 2026-09-10 22:19:38.006556737 +0000 UTC m=+84.802772192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c") : secret "alertmanager-main-tls" not found Sep 10 22:19:37.211983 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.211909 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b458d4d45-ngsd4"] Sep 10 22:19:37.214028 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.214013 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.216587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216373 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 22:19:37.216587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216373 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 22:19:37.216587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216430 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 22:19:37.216587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216378 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zzpsp\"" Sep 10 22:19:37.216587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216470 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 22:19:37.216587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216572 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 22:19:37.216959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216934 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 22:19:37.216959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.216934 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 22:19:37.221662 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.221617 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 22:19:37.224535 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.224512 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b458d4d45-ngsd4"] Sep 10 22:19:37.252833 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.252790 2577 generic.go:356] "Generic (PLEG): container finished" podID="4dcfaacc-843c-4e92-9570-bd0fbde357d3" containerID="3f7925140a7fce3fef7cad64adbf0446dc429bcb7067c4a97fe66c1d5b8b5354" exitCode=0 Sep 10 22:19:37.253219 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.252849 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7js62" event={"ID":"4dcfaacc-843c-4e92-9570-bd0fbde357d3","Type":"ContainerDied","Data":"3f7925140a7fce3fef7cad64adbf0446dc429bcb7067c4a97fe66c1d5b8b5354"} Sep 10 22:19:37.255458 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.255437 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" event={"ID":"a1ba225a-2b83-4110-be11-59a10739dac6","Type":"ContainerStarted","Data":"d0fc9c3bf527f2b50ab620a1dfe67a240cbd8f84a5856e0b222b882b8cc8f201"} Sep 10 22:19:37.295010 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.294956 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-47cr8" podStartSLOduration=1.212340766 podStartE2EDuration="2.294942155s" totalImagesPullingTime="1.082601389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:37.293112416 +0000 UTC m=+84.089327897" watchObservedRunningTime="2026-09-10 22:19:37.294942155 +0000 UTC m=+84.091157632" Sep 10 22:19:37.308797 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.308776 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-oauth-serving-cert\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.308891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.308806 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-config\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.308891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.308842 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-oauth-config\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.308891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.308875 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-serving-cert\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.308997 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.308910 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-trusted-ca-bundle\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.309027 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.308989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wxt2\" (UniqueName: \"kubernetes.io/projected/e15e9460-8b93-4089-8d8d-a93d09ef41b0-kube-api-access-2wxt2\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.309057 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.309030 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-service-ca\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.409974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-oauth-serving-cert\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410147 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-config\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410147 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410077 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-oauth-config\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410147 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410107 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-serving-cert\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-trusted-ca-bundle\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wxt2\" (UniqueName: \"kubernetes.io/projected/e15e9460-8b93-4089-8d8d-a93d09ef41b0-kube-api-access-2wxt2\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410432 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-service-ca\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410650 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410607 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-oauth-serving-cert\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.410985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.410960 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-config\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.411104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.411086 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-service-ca\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.411422 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.411402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-trusted-ca-bundle\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.412957 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.412928 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-oauth-config\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.413097 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.413079 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-serving-cert\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.418418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.418397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wxt2\" (UniqueName: \"kubernetes.io/projected/e15e9460-8b93-4089-8d8d-a93d09ef41b0-kube-api-access-2wxt2\") pod \"console-5b458d4d45-ngsd4\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.526153 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.526127 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:37.797127 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:37.797034 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b458d4d45-ngsd4"] Sep 10 22:19:37.800483 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:37.800455 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode15e9460_8b93_4089_8d8d_a93d09ef41b0.slice/crio-ba9dc4cce2d018d8e1fa14fbe00701feb214b6af744df1e0607f4e163e7dff0a WatchSource:0}: Error finding container ba9dc4cce2d018d8e1fa14fbe00701feb214b6af744df1e0607f4e163e7dff0a: Status 404 returned error can't find the container with id ba9dc4cce2d018d8e1fa14fbe00701feb214b6af744df1e0607f4e163e7dff0a Sep 10 22:19:38.015193 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.015152 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:38.017647 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.017628 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:38.038830 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.038800 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:19:38.124480 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.124447 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64966f776b-q8fdn"] Sep 10 22:19:38.127162 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.127144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.130228 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.130196 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 10 22:19:38.130343 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.130233 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 10 22:19:38.130343 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.130196 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 10 22:19:38.130487 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.130469 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 10 22:19:38.130571 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.130555 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 10 22:19:38.130709 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.130694 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-cgqqffq6im52s\"" Sep 10 22:19:38.130841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.130807 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kpzdw\"" Sep 10 22:19:38.142295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.142275 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64966f776b-q8fdn"] Sep 10 22:19:38.263019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.262986 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7js62" event={"ID":"4dcfaacc-843c-4e92-9570-bd0fbde357d3","Type":"ContainerStarted","Data":"56c5b2bb0558025b6bf1e323d26cfaa405721618c48ec456079be30ba4251e28"} Sep 10 22:19:38.263528 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.263047 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7js62" event={"ID":"4dcfaacc-843c-4e92-9570-bd0fbde357d3","Type":"ContainerStarted","Data":"6a53e670b07c715708cbb7e53c335c51426f7c17608c0980f1f911baba12746c"} Sep 10 22:19:38.264520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.264499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b458d4d45-ngsd4" event={"ID":"e15e9460-8b93-4089-8d8d-a93d09ef41b0","Type":"ContainerStarted","Data":"ba9dc4cce2d018d8e1fa14fbe00701feb214b6af744df1e0607f4e163e7dff0a"} Sep 10 22:19:38.281779 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.281723 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7js62" podStartSLOduration=1.814347743 podStartE2EDuration="3.281710319s" totalImagesPullingTime="1.395542657s" totalInitContainerRuntime="71.819919ms" isStatefulPod=true podCreationTimestamp="2026-09-10 22:19:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:38.279514783 +0000 UTC m=+85.075730260" watchObservedRunningTime="2026-09-10 22:19:38.281710319 +0000 UTC m=+85.077925796" Sep 10 22:19:38.317134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317105 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:19:38.317455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317425 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m49l4\" (UniqueName: \"kubernetes.io/projected/694690b8-b2a4-46c5-bad4-f2c313d3254b-kube-api-access-m49l4\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.317563 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317485 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.317563 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-grpc-tls\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.317672 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317586 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.317672 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/694690b8-b2a4-46c5-bad4-f2c313d3254b-metrics-client-ca\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.317784 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317681 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.317784 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317740 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-tls\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.317912 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.317789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.319623 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:38.319601 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c387fb1_35f5_423c_91fe_534c62ee0d3c.slice/crio-bc2a511236f609922666b85cb67aa3fb6a57e668985e106db866ca21a41a4046 WatchSource:0}: Error finding container bc2a511236f609922666b85cb67aa3fb6a57e668985e106db866ca21a41a4046: Status 404 returned error can't find the container with id bc2a511236f609922666b85cb67aa3fb6a57e668985e106db866ca21a41a4046 Sep 10 22:19:38.418268 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-tls\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.418417 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418307 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.418417 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m49l4\" (UniqueName: \"kubernetes.io/projected/694690b8-b2a4-46c5-bad4-f2c313d3254b-kube-api-access-m49l4\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.418547 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.418547 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-grpc-tls\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.418547 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.419124 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/694690b8-b2a4-46c5-bad4-f2c313d3254b-metrics-client-ca\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.419124 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.418844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.419739 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.419526 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/694690b8-b2a4-46c5-bad4-f2c313d3254b-metrics-client-ca\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.421476 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.421432 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.422167 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.422134 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-tls\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.422376 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.422350 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-grpc-tls\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.422459 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.422431 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.423001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.422948 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.423533 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.423513 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/694690b8-b2a4-46c5-bad4-f2c313d3254b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.426567 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.426546 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m49l4\" (UniqueName: \"kubernetes.io/projected/694690b8-b2a4-46c5-bad4-f2c313d3254b-kube-api-access-m49l4\") pod \"thanos-querier-64966f776b-q8fdn\" (UID: \"694690b8-b2a4-46c5-bad4-f2c313d3254b\") " pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.437574 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.437366 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:38.722176 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:38.722147 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64966f776b-q8fdn"] Sep 10 22:19:38.723695 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:38.723667 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694690b8_b2a4_46c5_bad4_f2c313d3254b.slice/crio-738bf3d1cdc87a3f37715a4f4f458d533fc2da1a444eb20c6049debc7df864ae WatchSource:0}: Error finding container 738bf3d1cdc87a3f37715a4f4f458d533fc2da1a444eb20c6049debc7df864ae: Status 404 returned error can't find the container with id 738bf3d1cdc87a3f37715a4f4f458d533fc2da1a444eb20c6049debc7df864ae Sep 10 22:19:39.154893 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.154649 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:19:39.271683 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.271647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" event={"ID":"694690b8-b2a4-46c5-bad4-f2c313d3254b","Type":"ContainerStarted","Data":"738bf3d1cdc87a3f37715a4f4f458d533fc2da1a444eb20c6049debc7df864ae"} Sep 10 22:19:39.275071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.274999 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"bc2a511236f609922666b85cb67aa3fb6a57e668985e106db866ca21a41a4046"} Sep 10 22:19:39.512662 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.512633 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-746fddfb69-zcrbl"] Sep 10 22:19:39.515909 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.515889 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.518231 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.518204 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Sep 10 22:19:39.518328 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.518255 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 22:19:39.518328 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.518289 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qbdj2\"" Sep 10 22:19:39.518451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.518291 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Sep 10 22:19:39.518451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.518256 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2sng49r3g313m\"" Sep 10 22:19:39.518834 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.518795 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Sep 10 22:19:39.527014 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.526989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-746fddfb69-zcrbl"] Sep 10 22:19:39.631566 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.631534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-client-ca-bundle\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.631730 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.631598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7a1fed85-cc3f-4595-8798-0c8fac83f1df-audit-log\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.631730 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.631622 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7a1fed85-cc3f-4595-8798-0c8fac83f1df-metrics-server-audit-profiles\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.631730 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.631713 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1fed85-cc3f-4595-8798-0c8fac83f1df-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.631930 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.631791 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhjg6\" (UniqueName: \"kubernetes.io/projected/7a1fed85-cc3f-4595-8798-0c8fac83f1df-kube-api-access-qhjg6\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.631930 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.631873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-secret-metrics-server-tls\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.632018 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.631939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-secret-metrics-server-client-certs\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1fed85-cc3f-4595-8798-0c8fac83f1df-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhjg6\" (UniqueName: \"kubernetes.io/projected/7a1fed85-cc3f-4595-8798-0c8fac83f1df-kube-api-access-qhjg6\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733276 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-secret-metrics-server-tls\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733539 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-secret-metrics-server-client-certs\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733539 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-client-ca-bundle\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733539 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7a1fed85-cc3f-4595-8798-0c8fac83f1df-audit-log\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733539 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733432 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7a1fed85-cc3f-4595-8798-0c8fac83f1df-metrics-server-audit-profiles\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.733980 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.733952 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a1fed85-cc3f-4595-8798-0c8fac83f1df-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.735120 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.734743 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7a1fed85-cc3f-4595-8798-0c8fac83f1df-metrics-server-audit-profiles\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.735120 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.735065 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7a1fed85-cc3f-4595-8798-0c8fac83f1df-audit-log\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.736809 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.736785 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-secret-metrics-server-client-certs\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.737547 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.737526 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-client-ca-bundle\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.737839 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.737792 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7a1fed85-cc3f-4595-8798-0c8fac83f1df-secret-metrics-server-tls\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.745375 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.745344 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhjg6\" (UniqueName: \"kubernetes.io/projected/7a1fed85-cc3f-4595-8798-0c8fac83f1df-kube-api-access-qhjg6\") pod \"metrics-server-746fddfb69-zcrbl\" (UID: \"7a1fed85-cc3f-4595-8798-0c8fac83f1df\") " pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.827146 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.827112 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:39.837441 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.837405 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-w652v"] Sep 10 22:19:39.840060 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.840038 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:39.842284 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.842199 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nw8qj\"" Sep 10 22:19:39.842470 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.842305 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 10 22:19:39.848492 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.848466 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-w652v"] Sep 10 22:19:39.935384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.935351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-w652v\" (UID: \"e7bc890e-db81-4dc4-877b-bef02e2bf7a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:39.935551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:39.935399 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9k4\" (UniqueName: \"kubernetes.io/projected/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-kube-api-access-vb9k4\") pod \"monitoring-plugin-5d85778b9d-w652v\" (UID: \"e7bc890e-db81-4dc4-877b-bef02e2bf7a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:40.036221 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:40.036182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-w652v\" (UID: \"e7bc890e-db81-4dc4-877b-bef02e2bf7a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:40.036370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:40.036233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9k4\" (UniqueName: \"kubernetes.io/projected/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-kube-api-access-vb9k4\") pod \"monitoring-plugin-5d85778b9d-w652v\" (UID: \"e7bc890e-db81-4dc4-877b-bef02e2bf7a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:40.036370 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:40.036349 2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 10 22:19:40.036479 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:19:40.036417 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-monitoring-plugin-cert podName:e7bc890e-db81-4dc4-877b-bef02e2bf7a1 nodeName:}" failed. No retries permitted until 2026-09-10 22:19:40.536394732 +0000 UTC m=+87.332610200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-w652v" (UID: "e7bc890e-db81-4dc4-877b-bef02e2bf7a1") : secret "monitoring-plugin-cert" not found Sep 10 22:19:40.045609 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:40.045581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9k4\" (UniqueName: \"kubernetes.io/projected/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-kube-api-access-vb9k4\") pod \"monitoring-plugin-5d85778b9d-w652v\" (UID: \"e7bc890e-db81-4dc4-877b-bef02e2bf7a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:40.282334 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:40.282294 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc"} Sep 10 22:19:40.540766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:40.540735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-w652v\" (UID: \"e7bc890e-db81-4dc4-877b-bef02e2bf7a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:40.543579 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:40.543546 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e7bc890e-db81-4dc4-877b-bef02e2bf7a1-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-w652v\" (UID: \"e7bc890e-db81-4dc4-877b-bef02e2bf7a1\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:40.752988 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:40.752951 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:41.291972 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:41.291938 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerID="7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc" exitCode=0 Sep 10 22:19:41.292334 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:41.292044 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc"} Sep 10 22:19:41.524166 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:41.524140 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-w652v"] Sep 10 22:19:41.526253 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:41.526224 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7bc890e_db81_4dc4_877b_bef02e2bf7a1.slice/crio-73be7771b4615ed85b022f88ab023125d7c85567ff69a7369f05e2aa2f5d6552 WatchSource:0}: Error finding container 73be7771b4615ed85b022f88ab023125d7c85567ff69a7369f05e2aa2f5d6552: Status 404 returned error can't find the container with id 73be7771b4615ed85b022f88ab023125d7c85567ff69a7369f05e2aa2f5d6552 Sep 10 22:19:41.529615 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:41.529594 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-746fddfb69-zcrbl"] Sep 10 22:19:41.532388 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:19:41.532367 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1fed85_cc3f_4595_8798_0c8fac83f1df.slice/crio-e0625df0bcc0bbabdcfbc7e7d74d7082a299a0a4c1b78adbf7ecc714b05c0792 WatchSource:0}: Error finding container e0625df0bcc0bbabdcfbc7e7d74d7082a299a0a4c1b78adbf7ecc714b05c0792: Status 404 returned error can't find the container with id e0625df0bcc0bbabdcfbc7e7d74d7082a299a0a4c1b78adbf7ecc714b05c0792 Sep 10 22:19:42.300704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:42.300660 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" event={"ID":"e7bc890e-db81-4dc4-877b-bef02e2bf7a1","Type":"ContainerStarted","Data":"73be7771b4615ed85b022f88ab023125d7c85567ff69a7369f05e2aa2f5d6552"} Sep 10 22:19:42.303346 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:42.303314 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b458d4d45-ngsd4" event={"ID":"e15e9460-8b93-4089-8d8d-a93d09ef41b0","Type":"ContainerStarted","Data":"a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d"} Sep 10 22:19:42.307091 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:42.307058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" event={"ID":"694690b8-b2a4-46c5-bad4-f2c313d3254b","Type":"ContainerStarted","Data":"768debadee4cb36ed900c5f7c72237983cd574d12114de25d4803cdf1928ead4"} Sep 10 22:19:42.307199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:42.307140 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" event={"ID":"694690b8-b2a4-46c5-bad4-f2c313d3254b","Type":"ContainerStarted","Data":"290ec12611d51722619bbbb53078238036e2ea664c3f724d37628bcf4ba57e85"} Sep 10 22:19:42.307199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:42.307184 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" event={"ID":"694690b8-b2a4-46c5-bad4-f2c313d3254b","Type":"ContainerStarted","Data":"c5eecd7ad6247f824f304dcb0d7e509bdbac91c5d39143fe478322cd021621af"} Sep 10 22:19:42.309558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:42.309532 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" event={"ID":"7a1fed85-cc3f-4595-8798-0c8fac83f1df","Type":"ContainerStarted","Data":"e0625df0bcc0bbabdcfbc7e7d74d7082a299a0a4c1b78adbf7ecc714b05c0792"} Sep 10 22:19:42.323643 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:42.321986 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b458d4d45-ngsd4" podStartSLOduration=1.873047111 podStartE2EDuration="5.32196639s" totalImagesPullingTime="3.448919279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:42.320854795 +0000 UTC m=+89.117070272" watchObservedRunningTime="2026-09-10 22:19:42.32196639 +0000 UTC m=+89.118181868" Sep 10 22:19:43.317290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:43.317247 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" event={"ID":"694690b8-b2a4-46c5-bad4-f2c313d3254b","Type":"ContainerStarted","Data":"d4ff40bf324f2197cccee68c5c382bcd4b10a0e470f217d84f93883d8d033f0a"} Sep 10 22:19:43.317692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:43.317336 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" event={"ID":"694690b8-b2a4-46c5-bad4-f2c313d3254b","Type":"ContainerStarted","Data":"c7a9ff327f87c2b7920ae599d29c461a8ebd78b5272f1a225590c74affcc2d27"} Sep 10 22:19:44.326021 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.325985 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39"} Sep 10 22:19:44.326437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.326114 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c"} Sep 10 22:19:44.326437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.326156 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222"} Sep 10 22:19:44.326437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.326200 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b"} Sep 10 22:19:44.326437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.326238 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7"} Sep 10 22:19:44.326437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.326274 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerStarted","Data":"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1"} Sep 10 22:19:44.329136 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.329110 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" event={"ID":"694690b8-b2a4-46c5-bad4-f2c313d3254b","Type":"ContainerStarted","Data":"517d021e6b383582ba838c7e9b68ec6361716085c02eb5abb33044ada3d50c93"} Sep 10 22:19:44.329298 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.329280 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:44.330711 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.330688 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" event={"ID":"7a1fed85-cc3f-4595-8798-0c8fac83f1df","Type":"ContainerStarted","Data":"6ff3affc0a3bcb839310095a46d655606c007b583e6c5a953dc52405818ee177"} Sep 10 22:19:44.332179 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.332156 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" event={"ID":"e7bc890e-db81-4dc4-877b-bef02e2bf7a1","Type":"ContainerStarted","Data":"cff9405d2bbbef49151a59153ff83c644ceab969d3c844762118764db6f543df"} Sep 10 22:19:44.332365 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.332347 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:44.337574 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.337555 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" Sep 10 22:19:44.358377 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.358333 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.811301535 podStartE2EDuration="8.3583173s" totalImagesPullingTime="3.52999735s" totalInitContainerRuntime="17.018415ms" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:44.357163842 +0000 UTC m=+91.153379318" watchObservedRunningTime="2026-09-10 22:19:44.3583173 +0000 UTC m=+91.154532781" Sep 10 22:19:44.379981 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.379925 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" podStartSLOduration=2.902703974 podStartE2EDuration="6.379906678s" totalImagesPullingTime="3.477202704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:44.378972429 +0000 UTC m=+91.175187907" watchObservedRunningTime="2026-09-10 22:19:44.379906678 +0000 UTC m=+91.176122156" Sep 10 22:19:44.396019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.395973 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" podStartSLOduration=3.244393748 podStartE2EDuration="5.395959937s" totalImagesPullingTime="2.151566189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:44.395357692 +0000 UTC m=+91.191573166" watchObservedRunningTime="2026-09-10 22:19:44.395959937 +0000 UTC m=+91.192175413" Sep 10 22:19:44.413208 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:44.413165 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-w652v" podStartSLOduration=3.258305776 podStartE2EDuration="5.413152373s" totalImagesPullingTime="2.154846597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:44.411077779 +0000 UTC m=+91.207293256" watchObservedRunningTime="2026-09-10 22:19:44.413152373 +0000 UTC m=+91.209367877" Sep 10 22:19:45.110052 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:45.110015 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b458d4d45-ngsd4"] Sep 10 22:19:47.417555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:47.417525 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c7d7d47c8-2k8vr"] Sep 10 22:19:47.526749 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:47.526712 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:19:50.342963 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:50.342937 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64966f776b-q8fdn" Sep 10 22:19:59.827694 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:59.827657 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:19:59.828218 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:19:59.827721 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:20:07.438077 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:07.438043 2577 generic.go:356] "Generic (PLEG): container finished" podID="13205e70-75c2-4340-a014-0f3d44963cbd" containerID="4bbb9572d06a72ec1bb3636ef04117081e0c216dd448f0b3777eefa21b79090d" exitCode=0 Sep 10 22:20:07.438396 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:07.438119 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" event={"ID":"13205e70-75c2-4340-a014-0f3d44963cbd","Type":"ContainerDied","Data":"4bbb9572d06a72ec1bb3636ef04117081e0c216dd448f0b3777eefa21b79090d"} Sep 10 22:20:07.438436 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:07.438420 2577 scope.go:117] "RemoveContainer" containerID="4bbb9572d06a72ec1bb3636ef04117081e0c216dd448f0b3777eefa21b79090d" Sep 10 22:20:08.444362 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:08.444326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-ndxsf" event={"ID":"13205e70-75c2-4340-a014-0f3d44963cbd","Type":"ContainerStarted","Data":"f847fcb22e7161d3f7434c169f665ebbbaa0ae0edff5732a00aa05176276c5ca"} Sep 10 22:20:10.129585 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.129546 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b458d4d45-ngsd4" podUID="e15e9460-8b93-4089-8d8d-a93d09ef41b0" containerName="console" containerID="cri-o://a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d" gracePeriod=15 Sep 10 22:20:10.403663 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.403598 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:20:10.453395 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.453363 2577 generic.go:356] "Generic (PLEG): container finished" podID="e15e9460-8b93-4089-8d8d-a93d09ef41b0" containerID="a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d" exitCode=0 Sep 10 22:20:10.453524 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.453410 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b458d4d45-ngsd4" Sep 10 22:20:10.453524 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.453442 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b458d4d45-ngsd4" event={"ID":"e15e9460-8b93-4089-8d8d-a93d09ef41b0","Type":"ContainerDied","Data":"a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d"} Sep 10 22:20:10.453524 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.453486 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b458d4d45-ngsd4" event={"ID":"e15e9460-8b93-4089-8d8d-a93d09ef41b0","Type":"ContainerDied","Data":"ba9dc4cce2d018d8e1fa14fbe00701feb214b6af744df1e0607f4e163e7dff0a"} Sep 10 22:20:10.453524 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.453506 2577 scope.go:117] "RemoveContainer" containerID="a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d" Sep 10 22:20:10.462018 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.462000 2577 scope.go:117] "RemoveContainer" containerID="a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d" Sep 10 22:20:10.462269 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:10.462249 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d\": container with ID starting with a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d not found: ID does not exist" containerID="a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d" Sep 10 22:20:10.462319 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.462275 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d"} err="failed to get container status \"a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d\": rpc error: code = NotFound desc = could not find container \"a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d\": container with ID starting with a977910b38f6039fac5b11c0fa20f017c8da234373b7e28a491415e333f8739d not found: ID does not exist" Sep 10 22:20:10.498602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498578 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-service-ca\") pod \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " Sep 10 22:20:10.498704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498616 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-oauth-serving-cert\") pod \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " Sep 10 22:20:10.498704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498637 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-oauth-config\") pod \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " Sep 10 22:20:10.498704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498668 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-serving-cert\") pod \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " Sep 10 22:20:10.498704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498685 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wxt2\" (UniqueName: \"kubernetes.io/projected/e15e9460-8b93-4089-8d8d-a93d09ef41b0-kube-api-access-2wxt2\") pod \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " Sep 10 22:20:10.498930 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498795 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-trusted-ca-bundle\") pod \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " Sep 10 22:20:10.498930 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498890 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-config\") pod \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\" (UID: \"e15e9460-8b93-4089-8d8d-a93d09ef41b0\") " Sep 10 22:20:10.499030 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.498920 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-service-ca" (OuterVolumeSpecName: "service-ca") pod "e15e9460-8b93-4089-8d8d-a93d09ef41b0" (UID: "e15e9460-8b93-4089-8d8d-a93d09ef41b0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:10.499389 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.499095 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e15e9460-8b93-4089-8d8d-a93d09ef41b0" (UID: "e15e9460-8b93-4089-8d8d-a93d09ef41b0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:10.499389 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.499310 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:10.499389 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.499331 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-oauth-serving-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:10.499389 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.499333 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-config" (OuterVolumeSpecName: "console-config") pod "e15e9460-8b93-4089-8d8d-a93d09ef41b0" (UID: "e15e9460-8b93-4089-8d8d-a93d09ef41b0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:10.499608 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.499384 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e15e9460-8b93-4089-8d8d-a93d09ef41b0" (UID: "e15e9460-8b93-4089-8d8d-a93d09ef41b0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:10.500987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.500965 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e15e9460-8b93-4089-8d8d-a93d09ef41b0" (UID: "e15e9460-8b93-4089-8d8d-a93d09ef41b0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:10.501484 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.501452 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e15e9460-8b93-4089-8d8d-a93d09ef41b0" (UID: "e15e9460-8b93-4089-8d8d-a93d09ef41b0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:10.501484 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.501474 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e15e9460-8b93-4089-8d8d-a93d09ef41b0-kube-api-access-2wxt2" (OuterVolumeSpecName: "kube-api-access-2wxt2") pod "e15e9460-8b93-4089-8d8d-a93d09ef41b0" (UID: "e15e9460-8b93-4089-8d8d-a93d09ef41b0"). InnerVolumeSpecName "kube-api-access-2wxt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:10.600013 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.599981 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-trusted-ca-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:10.600013 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.600008 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-config\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:10.600013 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.600017 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-oauth-config\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:10.600224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.600027 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e15e9460-8b93-4089-8d8d-a93d09ef41b0-console-serving-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:10.600224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.600037 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wxt2\" (UniqueName: \"kubernetes.io/projected/e15e9460-8b93-4089-8d8d-a93d09ef41b0-kube-api-access-2wxt2\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:10.791551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.791524 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b458d4d45-ngsd4"] Sep 10 22:20:10.804877 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:10.804854 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b458d4d45-ngsd4"] Sep 10 22:20:11.752886 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:11.752848 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e15e9460-8b93-4089-8d8d-a93d09ef41b0" path="/var/lib/kubelet/pods/e15e9460-8b93-4089-8d8d-a93d09ef41b0/volumes" Sep 10 22:20:12.437295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.437230 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" podUID="5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" containerName="registry" containerID="cri-o://c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8" gracePeriod=30 Sep 10 22:20:12.707994 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.707935 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:20:12.816497 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816471 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-ca-trust-extracted\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.816917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816532 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-certificates\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.816917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816575 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-installation-pull-secrets\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.816917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816599 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-image-registry-private-configuration\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.816917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816619 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-bound-sa-token\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.816917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816721 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-trusted-ca\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.816917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816787 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.816917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816887 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x754l\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-kube-api-access-x754l\") pod \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\" (UID: \"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a\") " Sep 10 22:20:12.817283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.816961 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:12.817283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.817149 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-certificates\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:12.817283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.817183 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:12.819490 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.819467 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:12.819608 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.819514 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:12.819849 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.819804 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-kube-api-access-x754l" (OuterVolumeSpecName: "kube-api-access-x754l") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "kube-api-access-x754l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:12.819849 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.819838 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:12.819974 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.819948 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:12.824682 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.824659 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" (UID: "5378dc3b-6b96-428f-abf9-05d5c9ae9e6a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:20:12.918397 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.918362 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x754l\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-kube-api-access-x754l\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:12.918397 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.918391 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-ca-trust-extracted\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:12.918607 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.918406 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-installation-pull-secrets\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:12.918607 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.918420 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-image-registry-private-configuration\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:12.918607 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.918433 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-bound-sa-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:12.918607 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.918445 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-trusted-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:12.918607 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:12.918457 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a-registry-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:13.468985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.468948 2577 generic.go:356] "Generic (PLEG): container finished" podID="5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" containerID="c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8" exitCode=0 Sep 10 22:20:13.469138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.468996 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" Sep 10 22:20:13.469138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.469026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" event={"ID":"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a","Type":"ContainerDied","Data":"c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8"} Sep 10 22:20:13.469138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.469071 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c7d7d47c8-2k8vr" event={"ID":"5378dc3b-6b96-428f-abf9-05d5c9ae9e6a","Type":"ContainerDied","Data":"f8f4de25e5163601ce124a04cc6c31dae9dc00ea033fcbea4a88bce6863be009"} Sep 10 22:20:13.469138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.469098 2577 scope.go:117] "RemoveContainer" containerID="c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8" Sep 10 22:20:13.478304 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.478283 2577 scope.go:117] "RemoveContainer" containerID="c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8" Sep 10 22:20:13.478567 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:13.478548 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8\": container with ID starting with c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8 not found: ID does not exist" containerID="c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8" Sep 10 22:20:13.478609 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.478575 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8"} err="failed to get container status \"c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8\": rpc error: code = NotFound desc = could not find container \"c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8\": container with ID starting with c1bf9676c17a322898f598192364f656d00a2eeb45bdf71702eff3ad51a8c5d8 not found: ID does not exist" Sep 10 22:20:13.489008 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.488986 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c7d7d47c8-2k8vr"] Sep 10 22:20:13.495448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.495421 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5c7d7d47c8-2k8vr"] Sep 10 22:20:13.747919 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:13.747847 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" path="/var/lib/kubelet/pods/5378dc3b-6b96-428f-abf9-05d5c9ae9e6a/volumes" Sep 10 22:20:17.486859 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:17.486807 2577 generic.go:356] "Generic (PLEG): container finished" podID="e9f0338c-f42f-4873-8ebe-7ae3d7541cda" containerID="678d4a1b1d6902ff27c47554ba19d62222c958ce71aa3c906b3a99a6c198ea03" exitCode=0 Sep 10 22:20:17.487290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:17.486866 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" event={"ID":"e9f0338c-f42f-4873-8ebe-7ae3d7541cda","Type":"ContainerDied","Data":"678d4a1b1d6902ff27c47554ba19d62222c958ce71aa3c906b3a99a6c198ea03"} Sep 10 22:20:17.487290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:17.487164 2577 scope.go:117] "RemoveContainer" containerID="678d4a1b1d6902ff27c47554ba19d62222c958ce71aa3c906b3a99a6c198ea03" Sep 10 22:20:18.493074 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:18.493037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-4gsm2" event={"ID":"e9f0338c-f42f-4873-8ebe-7ae3d7541cda","Type":"ContainerStarted","Data":"8cea5a68e4055ec779fce6b52178213efd0e2bc3d795597acd88db45cc95a4f8"} Sep 10 22:20:19.833280 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:19.833249 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:20:19.837104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:19.837079 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-746fddfb69-zcrbl" Sep 10 22:20:27.533643 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:27.533609 2577 generic.go:356] "Generic (PLEG): container finished" podID="099a0642-5254-4571-8da9-e6cce52ff27f" containerID="96032bc9e6362f8897bb4a1189e638d8818b18d6c68198cd5fee352de7438bbd" exitCode=0 Sep 10 22:20:27.534055 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:27.533684 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jv4bp" event={"ID":"099a0642-5254-4571-8da9-e6cce52ff27f","Type":"ContainerDied","Data":"96032bc9e6362f8897bb4a1189e638d8818b18d6c68198cd5fee352de7438bbd"} Sep 10 22:20:27.534099 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:27.534057 2577 scope.go:117] "RemoveContainer" containerID="96032bc9e6362f8897bb4a1189e638d8818b18d6c68198cd5fee352de7438bbd" Sep 10 22:20:28.539948 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:28.539913 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jv4bp" event={"ID":"099a0642-5254-4571-8da9-e6cce52ff27f","Type":"ContainerStarted","Data":"b8174a5cbfacefd219a72c0cdf75df431db5b4a697bc23ba7e34d838223afc18"} Sep 10 22:20:55.377535 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.377492 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:20:55.378002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.377749 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="alertmanager" containerID="cri-o://ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1" gracePeriod=120 Sep 10 22:20:55.378002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.377844 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-metric" containerID="cri-o://d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c" gracePeriod=120 Sep 10 22:20:55.378002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.377872 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-web" containerID="cri-o://572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b" gracePeriod=120 Sep 10 22:20:55.378002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.377872 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="config-reloader" containerID="cri-o://fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7" gracePeriod=120 Sep 10 22:20:55.378255 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.377998 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy" containerID="cri-o://9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222" gracePeriod=120 Sep 10 22:20:55.378255 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.377928 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="prom-label-proxy" containerID="cri-o://40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39" gracePeriod=120 Sep 10 22:20:55.655503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655422 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerID="40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39" exitCode=0 Sep 10 22:20:55.655503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655444 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerID="9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222" exitCode=0 Sep 10 22:20:55.655503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655450 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerID="fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7" exitCode=0 Sep 10 22:20:55.655503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655456 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerID="ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1" exitCode=0 Sep 10 22:20:55.655503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39"} Sep 10 22:20:55.655777 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655554 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222"} Sep 10 22:20:55.655777 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655581 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7"} Sep 10 22:20:55.655777 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:55.655606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1"} Sep 10 22:20:56.619273 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.619249 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:56.663697 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.663664 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerID="d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c" exitCode=0 Sep 10 22:20:56.663697 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.663692 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerID="572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b" exitCode=0 Sep 10 22:20:56.663985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.663741 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:56.663985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.663760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c"} Sep 10 22:20:56.663985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.663907 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b"} Sep 10 22:20:56.663985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.663959 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1c387fb1-35f5-423c-91fe-534c62ee0d3c","Type":"ContainerDied","Data":"bc2a511236f609922666b85cb67aa3fb6a57e668985e106db866ca21a41a4046"} Sep 10 22:20:56.663985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.663971 2577 scope.go:117] "RemoveContainer" containerID="40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39" Sep 10 22:20:56.669209 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669185 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-web-config\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669323 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669239 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-volume\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669323 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669274 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-metrics-client-ca\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669323 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669312 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-main-db\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669566 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669421 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-tls-assets\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669566 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669473 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669566 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669520 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-cluster-tls-config\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669566 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669554 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-web\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669768 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669608 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnnb9\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-kube-api-access-rnnb9\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669768 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669655 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-trusted-ca-bundle\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669768 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669695 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669768 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.669975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.669791 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-out\") pod \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\" (UID: \"1c387fb1-35f5-423c-91fe-534c62ee0d3c\") " Sep 10 22:20:56.670351 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.670327 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:20:56.670577 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.670549 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:56.671149 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.671125 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:20:56.674626 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.674318 2577 scope.go:117] "RemoveContainer" containerID="d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c" Sep 10 22:20:56.675154 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.675084 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-kube-api-access-rnnb9" (OuterVolumeSpecName: "kube-api-access-rnnb9") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "kube-api-access-rnnb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:56.676728 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.676693 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:56.677397 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.677373 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:20:56.677490 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.677421 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:56.677555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.677501 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:56.677704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.677674 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-volume" (OuterVolumeSpecName: "config-volume") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:56.677704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.677666 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-out" (OuterVolumeSpecName: "config-out") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:20:56.679226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.679190 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:56.680646 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.680625 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:56.687558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.687537 2577 scope.go:117] "RemoveContainer" containerID="9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222" Sep 10 22:20:56.687651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.687588 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-web-config" (OuterVolumeSpecName: "web-config") pod "1c387fb1-35f5-423c-91fe-534c62ee0d3c" (UID: "1c387fb1-35f5-423c-91fe-534c62ee0d3c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:20:56.696062 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.696043 2577 scope.go:117] "RemoveContainer" containerID="572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b" Sep 10 22:20:56.702636 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.702598 2577 scope.go:117] "RemoveContainer" containerID="fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7" Sep 10 22:20:56.710999 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.710982 2577 scope.go:117] "RemoveContainer" containerID="ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1" Sep 10 22:20:56.718475 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.718453 2577 scope.go:117] "RemoveContainer" containerID="7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc" Sep 10 22:20:56.725518 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.725497 2577 scope.go:117] "RemoveContainer" containerID="40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39" Sep 10 22:20:56.725793 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:56.725771 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39\": container with ID starting with 40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39 not found: ID does not exist" containerID="40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39" Sep 10 22:20:56.725877 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.725802 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39"} err="failed to get container status \"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39\": rpc error: code = NotFound desc = could not find container \"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39\": container with ID starting with 40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39 not found: ID does not exist" Sep 10 22:20:56.725877 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.725842 2577 scope.go:117] "RemoveContainer" containerID="d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c" Sep 10 22:20:56.726101 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:56.726084 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c\": container with ID starting with d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c not found: ID does not exist" containerID="d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c" Sep 10 22:20:56.726145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726108 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c"} err="failed to get container status \"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c\": rpc error: code = NotFound desc = could not find container \"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c\": container with ID starting with d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c not found: ID does not exist" Sep 10 22:20:56.726145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726124 2577 scope.go:117] "RemoveContainer" containerID="9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222" Sep 10 22:20:56.726363 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:56.726346 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222\": container with ID starting with 9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222 not found: ID does not exist" containerID="9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222" Sep 10 22:20:56.726410 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726370 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222"} err="failed to get container status \"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222\": rpc error: code = NotFound desc = could not find container \"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222\": container with ID starting with 9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222 not found: ID does not exist" Sep 10 22:20:56.726410 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726384 2577 scope.go:117] "RemoveContainer" containerID="572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b" Sep 10 22:20:56.726621 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:56.726604 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b\": container with ID starting with 572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b not found: ID does not exist" containerID="572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b" Sep 10 22:20:56.726663 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726625 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b"} err="failed to get container status \"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b\": rpc error: code = NotFound desc = could not find container \"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b\": container with ID starting with 572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b not found: ID does not exist" Sep 10 22:20:56.726663 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726639 2577 scope.go:117] "RemoveContainer" containerID="fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7" Sep 10 22:20:56.726901 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:56.726882 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7\": container with ID starting with fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7 not found: ID does not exist" containerID="fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7" Sep 10 22:20:56.726970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726903 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7"} err="failed to get container status \"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7\": rpc error: code = NotFound desc = could not find container \"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7\": container with ID starting with fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7 not found: ID does not exist" Sep 10 22:20:56.726970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.726917 2577 scope.go:117] "RemoveContainer" containerID="ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1" Sep 10 22:20:56.727173 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:56.727146 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1\": container with ID starting with ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1 not found: ID does not exist" containerID="ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1" Sep 10 22:20:56.727258 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727176 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1"} err="failed to get container status \"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1\": rpc error: code = NotFound desc = could not find container \"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1\": container with ID starting with ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1 not found: ID does not exist" Sep 10 22:20:56.727258 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727191 2577 scope.go:117] "RemoveContainer" containerID="7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc" Sep 10 22:20:56.727447 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:20:56.727430 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc\": container with ID starting with 7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc not found: ID does not exist" containerID="7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc" Sep 10 22:20:56.727483 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727452 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc"} err="failed to get container status \"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc\": rpc error: code = NotFound desc = could not find container \"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc\": container with ID starting with 7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc not found: ID does not exist" Sep 10 22:20:56.727483 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727469 2577 scope.go:117] "RemoveContainer" containerID="40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39" Sep 10 22:20:56.727695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727679 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39"} err="failed to get container status \"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39\": rpc error: code = NotFound desc = could not find container \"40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39\": container with ID starting with 40ad0831127ce7f5066b2bf68a2e0b89bfec0b07281d715e759108baf9f49e39 not found: ID does not exist" Sep 10 22:20:56.727695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727694 2577 scope.go:117] "RemoveContainer" containerID="d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c" Sep 10 22:20:56.727948 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727930 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c"} err="failed to get container status \"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c\": rpc error: code = NotFound desc = could not find container \"d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c\": container with ID starting with d92e89c9e14c15710b78d88f1f5988ba5bd3587f25ead522ba19de3de1a80a6c not found: ID does not exist" Sep 10 22:20:56.727948 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.727949 2577 scope.go:117] "RemoveContainer" containerID="9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222" Sep 10 22:20:56.728155 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728138 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222"} err="failed to get container status \"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222\": rpc error: code = NotFound desc = could not find container \"9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222\": container with ID starting with 9b99433c853fa24b3c6e892eb4dca962458af3604ce1d08146afc9626ddd7222 not found: ID does not exist" Sep 10 22:20:56.728212 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728156 2577 scope.go:117] "RemoveContainer" containerID="572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b" Sep 10 22:20:56.728369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728349 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b"} err="failed to get container status \"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b\": rpc error: code = NotFound desc = could not find container \"572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b\": container with ID starting with 572d8fcb33764dadfa12374fee84ace5ae7f5e86caf724ed7e603bc2f80ea12b not found: ID does not exist" Sep 10 22:20:56.728444 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728370 2577 scope.go:117] "RemoveContainer" containerID="fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7" Sep 10 22:20:56.728646 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728630 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7"} err="failed to get container status \"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7\": rpc error: code = NotFound desc = could not find container \"fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7\": container with ID starting with fc6b3c980570ee58e218f58dff6f9537b68df284be528f583d6b0363637f4eb7 not found: ID does not exist" Sep 10 22:20:56.728690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728648 2577 scope.go:117] "RemoveContainer" containerID="ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1" Sep 10 22:20:56.728865 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728849 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1"} err="failed to get container status \"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1\": rpc error: code = NotFound desc = could not find container \"ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1\": container with ID starting with ba10b3ea059cf13b8b75dee3112ea9d6bce23043430d44cf222cec1ee0c3f7d1 not found: ID does not exist" Sep 10 22:20:56.728932 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.728865 2577 scope.go:117] "RemoveContainer" containerID="7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc" Sep 10 22:20:56.729077 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.729063 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc"} err="failed to get container status \"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc\": rpc error: code = NotFound desc = could not find container \"7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc\": container with ID starting with 7e6369c3dd8595399d51287bf22979c96cff52d5a5629f1853b6423e8c58c3cc not found: ID does not exist" Sep 10 22:20:56.770979 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.770955 2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-out\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.770979 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.770977 2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-web-config\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.770986 2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-config-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.770994 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-metrics-client-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771004 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-main-db\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771012 2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-tls-assets\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771021 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771030 2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-cluster-tls-config\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771039 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771048 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnnb9\" (UniqueName: \"kubernetes.io/projected/1c387fb1-35f5-423c-91fe-534c62ee0d3c-kube-api-access-rnnb9\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771057 2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c387fb1-35f5-423c-91fe-534c62ee0d3c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771066 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.771089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.771074 2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1c387fb1-35f5-423c-91fe-534c62ee0d3c-secret-alertmanager-main-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:20:56.986923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.986896 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:20:56.990926 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:56.990904 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:20:57.019628 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019603 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:20:57.019915 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019902 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e15e9460-8b93-4089-8d8d-a93d09ef41b0" containerName="console" Sep 10 22:20:57.019961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019917 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e15e9460-8b93-4089-8d8d-a93d09ef41b0" containerName="console" Sep 10 22:20:57.019961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019926 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy" Sep 10 22:20:57.019961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019932 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy" Sep 10 22:20:57.019961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019940 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="prom-label-proxy" Sep 10 22:20:57.019961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019946 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="prom-label-proxy" Sep 10 22:20:57.019961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019955 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-web" Sep 10 22:20:57.019961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019961 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-web" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019967 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-metric" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019972 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-metric" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019980 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="init-config-reloader" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019985 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="init-config-reloader" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019990 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="config-reloader" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.019995 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="config-reloader" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020003 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" containerName="registry" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020009 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" containerName="registry" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020016 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="alertmanager" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020020 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="alertmanager" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020075 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-metric" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020083 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e15e9460-8b93-4089-8d8d-a93d09ef41b0" containerName="console" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020090 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="config-reloader" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020096 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="prom-label-proxy" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020102 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="alertmanager" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020107 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy-web" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020113 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5378dc3b-6b96-428f-abf9-05d5c9ae9e6a" containerName="registry" Sep 10 22:20:57.020157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.020119 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" containerName="kube-rbac-proxy" Sep 10 22:20:57.023759 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.023744 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.026145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026125 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Sep 10 22:20:57.026325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026310 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Sep 10 22:20:57.026402 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026343 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-d22jk\"" Sep 10 22:20:57.026402 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026373 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Sep 10 22:20:57.026522 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026408 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Sep 10 22:20:57.026578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026539 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Sep 10 22:20:57.026653 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026637 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Sep 10 22:20:57.026759 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026742 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Sep 10 22:20:57.026759 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.026755 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Sep 10 22:20:57.031453 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.031434 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Sep 10 22:20:57.040191 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.040170 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:20:57.072753 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072721 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.072933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072761 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-config-volume\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.072933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072790 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/02c61abd-a483-4bbd-bdfd-d982c9003a00-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.072933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.072933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psclq\" (UniqueName: \"kubernetes.io/projected/02c61abd-a483-4bbd-bdfd-d982c9003a00-kube-api-access-psclq\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.072933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072885 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.072933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02c61abd-a483-4bbd-bdfd-d982c9003a00-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.072933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02c61abd-a483-4bbd-bdfd-d982c9003a00-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.073208 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072947 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.073208 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.072978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-web-config\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.073208 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.073022 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.073208 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.073048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02c61abd-a483-4bbd-bdfd-d982c9003a00-config-out\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.073208 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.073065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02c61abd-a483-4bbd-bdfd-d982c9003a00-tls-assets\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174233 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174194 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174233 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02c61abd-a483-4bbd-bdfd-d982c9003a00-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02c61abd-a483-4bbd-bdfd-d982c9003a00-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174289 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174311 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-web-config\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02c61abd-a483-4bbd-bdfd-d982c9003a00-config-out\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02c61abd-a483-4bbd-bdfd-d982c9003a00-tls-assets\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174645 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174876 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-config-volume\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174876 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174752 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/02c61abd-a483-4bbd-bdfd-d982c9003a00-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174876 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174787 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.174876 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.174808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psclq\" (UniqueName: \"kubernetes.io/projected/02c61abd-a483-4bbd-bdfd-d982c9003a00-kube-api-access-psclq\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.175267 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.175156 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02c61abd-a483-4bbd-bdfd-d982c9003a00-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.175267 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.175254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02c61abd-a483-4bbd-bdfd-d982c9003a00-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.175532 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.175505 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/02c61abd-a483-4bbd-bdfd-d982c9003a00-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.177467 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.177436 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/02c61abd-a483-4bbd-bdfd-d982c9003a00-tls-assets\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.177920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.177857 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.178027 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.177958 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.178027 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.178010 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-config-volume\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.178138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.178123 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.178197 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.178180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/02c61abd-a483-4bbd-bdfd-d982c9003a00-config-out\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.178559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.178537 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.179266 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.179243 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-web-config\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.179584 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.179565 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/02c61abd-a483-4bbd-bdfd-d982c9003a00-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.182167 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.182149 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psclq\" (UniqueName: \"kubernetes.io/projected/02c61abd-a483-4bbd-bdfd-d982c9003a00-kube-api-access-psclq\") pod \"alertmanager-main-0\" (UID: \"02c61abd-a483-4bbd-bdfd-d982c9003a00\") " pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.333222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.333185 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Sep 10 22:20:57.611533 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.611506 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Sep 10 22:20:57.613493 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:20:57.613463 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02c61abd_a483_4bbd_bdfd_d982c9003a00.slice/crio-a912068829d06a72698487f80047e0ef3f6336e2a50deb5ca79be3a57341dac4 WatchSource:0}: Error finding container a912068829d06a72698487f80047e0ef3f6336e2a50deb5ca79be3a57341dac4: Status 404 returned error can't find the container with id a912068829d06a72698487f80047e0ef3f6336e2a50deb5ca79be3a57341dac4 Sep 10 22:20:57.669849 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.669803 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerStarted","Data":"a912068829d06a72698487f80047e0ef3f6336e2a50deb5ca79be3a57341dac4"} Sep 10 22:20:57.749173 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:57.749142 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c387fb1-35f5-423c-91fe-534c62ee0d3c" path="/var/lib/kubelet/pods/1c387fb1-35f5-423c-91fe-534c62ee0d3c/volumes" Sep 10 22:20:58.676845 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:58.676793 2577 generic.go:356] "Generic (PLEG): container finished" podID="02c61abd-a483-4bbd-bdfd-d982c9003a00" containerID="433aadf438b91fd2b479ba8ad08addd3e79e2959766d16fd0f41861374513922" exitCode=0 Sep 10 22:20:58.677294 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:58.676859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerDied","Data":"433aadf438b91fd2b479ba8ad08addd3e79e2959766d16fd0f41861374513922"} Sep 10 22:20:59.684689 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:59.684656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerStarted","Data":"e3ba653798a2ad57d9c277551a953f0dbc31443043d49b9f88fa21d40f7a0744"} Sep 10 22:20:59.685074 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:59.684720 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerStarted","Data":"c03bcc221cc8db3102968803d248f8e0c4c44ad205c89f7e9248508dcb126a5c"} Sep 10 22:20:59.685074 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:59.684747 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerStarted","Data":"c4a1b6f1c0761addbe9bdc72e07ede2e88459e8220f0f5e84f101fbe3fe5975b"} Sep 10 22:20:59.685074 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:59.684770 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerStarted","Data":"f8d3326c5717c0544ac2141c7612385064c0f52a287e96880d729a1aa3f49bfd"} Sep 10 22:20:59.685074 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:59.684795 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerStarted","Data":"a46c83b535342caa1b22bd84c49e17b7e853d39fd585b9ce7e84a39c952a26ff"} Sep 10 22:20:59.685074 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:59.684846 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"02c61abd-a483-4bbd-bdfd-d982c9003a00","Type":"ContainerStarted","Data":"4d2db65da76446335214a8525c99c3fb3fbf1f7ea46c9d5b8191368fb522deb7"} Sep 10 22:20:59.713216 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:20:59.713167 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.697360427 podStartE2EDuration="3.713153771s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.793344ms" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:59.711085805 +0000 UTC m=+166.507301283" watchObservedRunningTime="2026-09-10 22:20:59.713153771 +0000 UTC m=+166.509369298" Sep 10 22:22:02.127884 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.127848 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv"] Sep 10 22:22:02.131423 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.131400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.134069 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.134045 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lfkg2\"" Sep 10 22:22:02.134203 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.134126 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 10 22:22:02.134203 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.134176 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 10 22:22:02.147623 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.147596 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv"] Sep 10 22:22:02.181340 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.181298 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfkqm\" (UniqueName: \"kubernetes.io/projected/9c4ccade-6c44-44ff-a875-600d42c528c8-kube-api-access-qfkqm\") pod \"cert-manager-operator-controller-manager-55c444bf86-9bhwv\" (UID: \"9c4ccade-6c44-44ff-a875-600d42c528c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.181340 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.181348 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c4ccade-6c44-44ff-a875-600d42c528c8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9bhwv\" (UID: \"9c4ccade-6c44-44ff-a875-600d42c528c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.282655 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.282612 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfkqm\" (UniqueName: \"kubernetes.io/projected/9c4ccade-6c44-44ff-a875-600d42c528c8-kube-api-access-qfkqm\") pod \"cert-manager-operator-controller-manager-55c444bf86-9bhwv\" (UID: \"9c4ccade-6c44-44ff-a875-600d42c528c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.282655 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.282658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c4ccade-6c44-44ff-a875-600d42c528c8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9bhwv\" (UID: \"9c4ccade-6c44-44ff-a875-600d42c528c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.283048 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.283031 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c4ccade-6c44-44ff-a875-600d42c528c8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-9bhwv\" (UID: \"9c4ccade-6c44-44ff-a875-600d42c528c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.297075 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.297044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfkqm\" (UniqueName: \"kubernetes.io/projected/9c4ccade-6c44-44ff-a875-600d42c528c8-kube-api-access-qfkqm\") pod \"cert-manager-operator-controller-manager-55c444bf86-9bhwv\" (UID: \"9c4ccade-6c44-44ff-a875-600d42c528c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.441028 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.440937 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" Sep 10 22:22:02.715327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.715249 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv"] Sep 10 22:22:02.718846 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:22:02.718798 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4ccade_6c44_44ff_a875_600d42c528c8.slice/crio-f1b5f4969b6ec017a42c61a478702fd479aab0fcbde1973941dc312fa784d768 WatchSource:0}: Error finding container f1b5f4969b6ec017a42c61a478702fd479aab0fcbde1973941dc312fa784d768: Status 404 returned error can't find the container with id f1b5f4969b6ec017a42c61a478702fd479aab0fcbde1973941dc312fa784d768 Sep 10 22:22:02.932316 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:02.932278 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" event={"ID":"9c4ccade-6c44-44ff-a875-600d42c528c8","Type":"ContainerStarted","Data":"f1b5f4969b6ec017a42c61a478702fd479aab0fcbde1973941dc312fa784d768"} Sep 10 22:22:05.952577 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:05.952538 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" event={"ID":"9c4ccade-6c44-44ff-a875-600d42c528c8","Type":"ContainerStarted","Data":"d5ab2388f78b73a0fa5ac4f9f83b97ccf56058f4794659f8a48362128def0667"} Sep 10 22:22:05.975693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:05.975405 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-9bhwv" podStartSLOduration=1.205940755 podStartE2EDuration="3.97538763s" totalImagesPullingTime="2.769446875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:22:05.974034877 +0000 UTC m=+232.770250367" watchObservedRunningTime="2026-09-10 22:22:05.97538763 +0000 UTC m=+232.771603107" Sep 10 22:22:06.828972 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.828932 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44"] Sep 10 22:22:06.832427 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.832403 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:06.834493 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.834476 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 22:22:06.834565 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.834537 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pmzck\"" Sep 10 22:22:06.834996 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.834978 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 22:22:06.842108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.842090 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44"] Sep 10 22:22:06.926500 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.926444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:06.926500 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.926486 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sccng\" (UniqueName: \"kubernetes.io/projected/e28fee48-db1a-42ba-9bad-c8ef6f802b33-kube-api-access-sccng\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:06.926674 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:06.926512 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.027790 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.027761 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.028203 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.027852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sccng\" (UniqueName: \"kubernetes.io/projected/e28fee48-db1a-42ba-9bad-c8ef6f802b33-kube-api-access-sccng\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.028203 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.027892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.028300 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.028203 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.028300 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.028216 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.035718 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.035696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sccng\" (UniqueName: \"kubernetes.io/projected/e28fee48-db1a-42ba-9bad-c8ef6f802b33-kube-api-access-sccng\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.142258 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.142201 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:07.416191 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.416168 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44"] Sep 10 22:22:07.418225 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:22:07.418193 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28fee48_db1a_42ba_9bad_c8ef6f802b33.slice/crio-aee861b12f03417f5a561f731ae74a70618c28ddfea63fbf7865021ee29ebf29 WatchSource:0}: Error finding container aee861b12f03417f5a561f731ae74a70618c28ddfea63fbf7865021ee29ebf29: Status 404 returned error can't find the container with id aee861b12f03417f5a561f731ae74a70618c28ddfea63fbf7865021ee29ebf29 Sep 10 22:22:07.962695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:07.962652 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" event={"ID":"e28fee48-db1a-42ba-9bad-c8ef6f802b33","Type":"ContainerStarted","Data":"aee861b12f03417f5a561f731ae74a70618c28ddfea63fbf7865021ee29ebf29"} Sep 10 22:22:13.182615 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.182574 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk"] Sep 10 22:22:13.187929 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.187890 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.190267 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.190239 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 10 22:22:13.190392 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.190330 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ctwfs\"" Sep 10 22:22:13.190974 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.190951 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 10 22:22:13.194019 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.193998 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk"] Sep 10 22:22:13.282421 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.282387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmr8\" (UniqueName: \"kubernetes.io/projected/e1c134fc-e94f-4bb8-8068-39598cf0b006-kube-api-access-bhmr8\") pod \"cert-manager-cainjector-559cdd8c87-jfhsk\" (UID: \"e1c134fc-e94f-4bb8-8068-39598cf0b006\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.282584 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.282453 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1c134fc-e94f-4bb8-8068-39598cf0b006-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jfhsk\" (UID: \"e1c134fc-e94f-4bb8-8068-39598cf0b006\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.383704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.383665 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmr8\" (UniqueName: \"kubernetes.io/projected/e1c134fc-e94f-4bb8-8068-39598cf0b006-kube-api-access-bhmr8\") pod \"cert-manager-cainjector-559cdd8c87-jfhsk\" (UID: \"e1c134fc-e94f-4bb8-8068-39598cf0b006\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.383892 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.383746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1c134fc-e94f-4bb8-8068-39598cf0b006-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jfhsk\" (UID: \"e1c134fc-e94f-4bb8-8068-39598cf0b006\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.392675 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.392646 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1c134fc-e94f-4bb8-8068-39598cf0b006-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jfhsk\" (UID: \"e1c134fc-e94f-4bb8-8068-39598cf0b006\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.393010 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.392978 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmr8\" (UniqueName: \"kubernetes.io/projected/e1c134fc-e94f-4bb8-8068-39598cf0b006-kube-api-access-bhmr8\") pod \"cert-manager-cainjector-559cdd8c87-jfhsk\" (UID: \"e1c134fc-e94f-4bb8-8068-39598cf0b006\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.509689 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.509667 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" Sep 10 22:22:13.779226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.779203 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk"] Sep 10 22:22:13.781070 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:22:13.781046 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1c134fc_e94f_4bb8_8068_39598cf0b006.slice/crio-5dcd8277e21e21859369c70af23f279ac1e94c6cb91ff45f1385977d954f23da WatchSource:0}: Error finding container 5dcd8277e21e21859369c70af23f279ac1e94c6cb91ff45f1385977d954f23da: Status 404 returned error can't find the container with id 5dcd8277e21e21859369c70af23f279ac1e94c6cb91ff45f1385977d954f23da Sep 10 22:22:13.991088 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.991051 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" event={"ID":"e1c134fc-e94f-4bb8-8068-39598cf0b006","Type":"ContainerStarted","Data":"5dcd8277e21e21859369c70af23f279ac1e94c6cb91ff45f1385977d954f23da"} Sep 10 22:22:13.992511 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.992483 2577 generic.go:356] "Generic (PLEG): container finished" podID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerID="2cb707ac2d7d370d668025b9270f5181441674bc9c25090da92747a170f48066" exitCode=0 Sep 10 22:22:13.992631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:13.992568 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" event={"ID":"e28fee48-db1a-42ba-9bad-c8ef6f802b33","Type":"ContainerDied","Data":"2cb707ac2d7d370d668025b9270f5181441674bc9c25090da92747a170f48066"} Sep 10 22:22:17.010722 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.010683 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" event={"ID":"e1c134fc-e94f-4bb8-8068-39598cf0b006","Type":"ContainerStarted","Data":"671062956a74d133e3f8d4bcf820c2013a60c6cc52d936c2eeb6678044f8f819"} Sep 10 22:22:17.027845 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.027782 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jfhsk" podStartSLOduration=1.436121374 podStartE2EDuration="4.027766803s" totalImagesPullingTime="2.591645429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:22:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:22:17.026117281 +0000 UTC m=+243.822332760" watchObservedRunningTime="2026-09-10 22:22:17.027766803 +0000 UTC m=+243.823982279" Sep 10 22:22:17.071094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.071062 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s97jx"] Sep 10 22:22:17.074328 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.074295 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.076874 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.076854 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4hlk6\"" Sep 10 22:22:17.087749 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.087728 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s97jx"] Sep 10 22:22:17.216680 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.216644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba1c107c-82a5-445f-92f3-af7f8ae9c6ac-bound-sa-token\") pod \"cert-manager-55b4d954d5-s97jx\" (UID: \"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac\") " pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.216680 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.216686 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxpg7\" (UniqueName: \"kubernetes.io/projected/ba1c107c-82a5-445f-92f3-af7f8ae9c6ac-kube-api-access-lxpg7\") pod \"cert-manager-55b4d954d5-s97jx\" (UID: \"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac\") " pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.318102 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.318065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba1c107c-82a5-445f-92f3-af7f8ae9c6ac-bound-sa-token\") pod \"cert-manager-55b4d954d5-s97jx\" (UID: \"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac\") " pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.318249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.318106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxpg7\" (UniqueName: \"kubernetes.io/projected/ba1c107c-82a5-445f-92f3-af7f8ae9c6ac-kube-api-access-lxpg7\") pod \"cert-manager-55b4d954d5-s97jx\" (UID: \"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac\") " pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.325955 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.325927 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba1c107c-82a5-445f-92f3-af7f8ae9c6ac-bound-sa-token\") pod \"cert-manager-55b4d954d5-s97jx\" (UID: \"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac\") " pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.326060 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.325934 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxpg7\" (UniqueName: \"kubernetes.io/projected/ba1c107c-82a5-445f-92f3-af7f8ae9c6ac-kube-api-access-lxpg7\") pod \"cert-manager-55b4d954d5-s97jx\" (UID: \"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac\") " pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.383874 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.383841 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-s97jx" Sep 10 22:22:17.653159 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:17.653135 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-s97jx"] Sep 10 22:22:17.655831 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:22:17.655783 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba1c107c_82a5_445f_92f3_af7f8ae9c6ac.slice/crio-f0b066d7e088757c2de602b5193185335a56c2ad7a32b44286f7ad08654a7c41 WatchSource:0}: Error finding container f0b066d7e088757c2de602b5193185335a56c2ad7a32b44286f7ad08654a7c41: Status 404 returned error can't find the container with id f0b066d7e088757c2de602b5193185335a56c2ad7a32b44286f7ad08654a7c41 Sep 10 22:22:18.021303 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:18.021201 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s97jx" event={"ID":"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac","Type":"ContainerStarted","Data":"7e5ca38fa8b4b5e05016dbe15f2a719fc6f044953f30ed805b91d5023589cb8b"} Sep 10 22:22:18.021303 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:18.021267 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-s97jx" event={"ID":"ba1c107c-82a5-445f-92f3-af7f8ae9c6ac","Type":"ContainerStarted","Data":"f0b066d7e088757c2de602b5193185335a56c2ad7a32b44286f7ad08654a7c41"} Sep 10 22:22:18.036678 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:18.036580 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-s97jx" podStartSLOduration=1.036561054 podStartE2EDuration="1.036561054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:22:18.035759114 +0000 UTC m=+244.831974593" watchObservedRunningTime="2026-09-10 22:22:18.036561054 +0000 UTC m=+244.832776531" Sep 10 22:22:20.033786 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:20.033746 2577 generic.go:356] "Generic (PLEG): container finished" podID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerID="5fb44cbe975378377ae8fee00699ceb368d23cde26e8cf21adb46d14e9dc618c" exitCode=0 Sep 10 22:22:20.034214 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:20.033798 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" event={"ID":"e28fee48-db1a-42ba-9bad-c8ef6f802b33","Type":"ContainerDied","Data":"5fb44cbe975378377ae8fee00699ceb368d23cde26e8cf21adb46d14e9dc618c"} Sep 10 22:22:27.074902 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:27.074869 2577 generic.go:356] "Generic (PLEG): container finished" podID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerID="68d2fcc27695e2d6979fcaf96b635b4f183f13719b712db9bb80d6d3ee8395d6" exitCode=0 Sep 10 22:22:27.075309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:27.074952 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" event={"ID":"e28fee48-db1a-42ba-9bad-c8ef6f802b33","Type":"ContainerDied","Data":"68d2fcc27695e2d6979fcaf96b635b4f183f13719b712db9bb80d6d3ee8395d6"} Sep 10 22:22:28.203179 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.203156 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:28.316378 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.316351 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-bundle\") pod \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " Sep 10 22:22:28.316534 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.316426 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-util\") pod \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " Sep 10 22:22:28.316534 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.316457 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sccng\" (UniqueName: \"kubernetes.io/projected/e28fee48-db1a-42ba-9bad-c8ef6f802b33-kube-api-access-sccng\") pod \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\" (UID: \"e28fee48-db1a-42ba-9bad-c8ef6f802b33\") " Sep 10 22:22:28.316748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.316723 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-bundle" (OuterVolumeSpecName: "bundle") pod "e28fee48-db1a-42ba-9bad-c8ef6f802b33" (UID: "e28fee48-db1a-42ba-9bad-c8ef6f802b33"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:22:28.318757 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.318729 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28fee48-db1a-42ba-9bad-c8ef6f802b33-kube-api-access-sccng" (OuterVolumeSpecName: "kube-api-access-sccng") pod "e28fee48-db1a-42ba-9bad-c8ef6f802b33" (UID: "e28fee48-db1a-42ba-9bad-c8ef6f802b33"). InnerVolumeSpecName "kube-api-access-sccng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:22:28.320543 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.320515 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-util" (OuterVolumeSpecName: "util") pod "e28fee48-db1a-42ba-9bad-c8ef6f802b33" (UID: "e28fee48-db1a-42ba-9bad-c8ef6f802b33"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:22:28.417762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.417704 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-util\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:22:28.417762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.417728 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sccng\" (UniqueName: \"kubernetes.io/projected/e28fee48-db1a-42ba-9bad-c8ef6f802b33-kube-api-access-sccng\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:22:28.417762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:28.417740 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e28fee48-db1a-42ba-9bad-c8ef6f802b33-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:22:29.085911 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:29.085880 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" Sep 10 22:22:29.086068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:29.085880 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftct44" event={"ID":"e28fee48-db1a-42ba-9bad-c8ef6f802b33","Type":"ContainerDied","Data":"aee861b12f03417f5a561f731ae74a70618c28ddfea63fbf7865021ee29ebf29"} Sep 10 22:22:29.086068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:29.085996 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aee861b12f03417f5a561f731ae74a70618c28ddfea63fbf7865021ee29ebf29" Sep 10 22:22:54.359473 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359438 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9"] Sep 10 22:22:54.360047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359808 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerName="pull" Sep 10 22:22:54.360047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359847 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerName="pull" Sep 10 22:22:54.360047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359869 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerName="util" Sep 10 22:22:54.360047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359876 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerName="util" Sep 10 22:22:54.360047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359883 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerName="extract" Sep 10 22:22:54.360047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359893 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerName="extract" Sep 10 22:22:54.360047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.359962 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28fee48-db1a-42ba-9bad-c8ef6f802b33" containerName="extract" Sep 10 22:22:54.362496 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.362475 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.364482 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.364440 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 22:22:54.365116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.365096 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 22:22:54.365231 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.365140 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pmzck\"" Sep 10 22:22:54.371762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.371737 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9"] Sep 10 22:22:54.427132 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.427102 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.427275 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.427139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.427275 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.427161 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lghd2\" (UniqueName: \"kubernetes.io/projected/aac9e13d-adb9-4a90-991c-1453766a0a98-kube-api-access-lghd2\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.527980 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.527949 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.527980 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.527983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.528143 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.528003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lghd2\" (UniqueName: \"kubernetes.io/projected/aac9e13d-adb9-4a90-991c-1453766a0a98-kube-api-access-lghd2\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.528322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.528303 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.528391 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.528372 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.538847 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.538802 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lghd2\" (UniqueName: \"kubernetes.io/projected/aac9e13d-adb9-4a90-991c-1453766a0a98-kube-api-access-lghd2\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.672845 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.672761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:54.942459 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:54.942434 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9"] Sep 10 22:22:54.944697 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:22:54.944665 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac9e13d_adb9_4a90_991c_1453766a0a98.slice/crio-6e103c94a7f669ff4f91d3277c2a492e7b2b515049954b924ec194dd0dd98ef7 WatchSource:0}: Error finding container 6e103c94a7f669ff4f91d3277c2a492e7b2b515049954b924ec194dd0dd98ef7: Status 404 returned error can't find the container with id 6e103c94a7f669ff4f91d3277c2a492e7b2b515049954b924ec194dd0dd98ef7 Sep 10 22:22:55.203855 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:55.203746 2577 generic.go:356] "Generic (PLEG): container finished" podID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerID="68a6f1e23bc761d9a94c4aba2a2ec75b74f1ca8b0333ef4a968be0c95237570c" exitCode=0 Sep 10 22:22:55.203855 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:55.203790 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" event={"ID":"aac9e13d-adb9-4a90-991c-1453766a0a98","Type":"ContainerDied","Data":"68a6f1e23bc761d9a94c4aba2a2ec75b74f1ca8b0333ef4a968be0c95237570c"} Sep 10 22:22:55.204091 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:55.203871 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" event={"ID":"aac9e13d-adb9-4a90-991c-1453766a0a98","Type":"ContainerStarted","Data":"6e103c94a7f669ff4f91d3277c2a492e7b2b515049954b924ec194dd0dd98ef7"} Sep 10 22:22:56.287912 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.287891 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft"] Sep 10 22:22:56.300168 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.300148 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.304292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.304271 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 22:22:56.304531 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.304512 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 22:22:56.304929 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.304894 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qk85w\"" Sep 10 22:22:56.305084 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.305068 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 22:22:56.305152 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.305140 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 22:22:56.314657 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.314635 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft"] Sep 10 22:22:56.342690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.342650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae81311b-cd92-434b-be58-71192a85ab62-manifests\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.342690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.342685 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnj59\" (UniqueName: \"kubernetes.io/projected/ae81311b-cd92-434b-be58-71192a85ab62-kube-api-access-tnj59\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.342880 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.342706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81311b-cd92-434b-be58-71192a85ab62-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.342880 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.342741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae81311b-cd92-434b-be58-71192a85ab62-tmp\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.342880 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.342808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81311b-cd92-434b-be58-71192a85ab62-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.444274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.444214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae81311b-cd92-434b-be58-71192a85ab62-manifests\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.444274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.444245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnj59\" (UniqueName: \"kubernetes.io/projected/ae81311b-cd92-434b-be58-71192a85ab62-kube-api-access-tnj59\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.444274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.444265 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81311b-cd92-434b-be58-71192a85ab62-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.444502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.444292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae81311b-cd92-434b-be58-71192a85ab62-tmp\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.444502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.444429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81311b-cd92-434b-be58-71192a85ab62-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.444617 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.444598 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ae81311b-cd92-434b-be58-71192a85ab62-manifests\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.446668 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.446641 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae81311b-cd92-434b-be58-71192a85ab62-tmp\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.446757 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.446742 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ae81311b-cd92-434b-be58-71192a85ab62-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.446803 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.446744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ae81311b-cd92-434b-be58-71192a85ab62-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.452608 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.452588 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnj59\" (UniqueName: \"kubernetes.io/projected/ae81311b-cd92-434b-be58-71192a85ab62-kube-api-access-tnj59\") pod \"opendatahub-operator-controller-manager-7c969df88c-vr5ft\" (UID: \"ae81311b-cd92-434b-be58-71192a85ab62\") " pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.611115 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.611080 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:22:56.914788 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:56.914762 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft"] Sep 10 22:22:56.916846 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:22:56.916793 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae81311b_cd92_434b_be58_71192a85ab62.slice/crio-53898a741ddbcebdd4a4920e3ea7d6d816c6c0b62081fcda34e9118ee0d14b5e WatchSource:0}: Error finding container 53898a741ddbcebdd4a4920e3ea7d6d816c6c0b62081fcda34e9118ee0d14b5e: Status 404 returned error can't find the container with id 53898a741ddbcebdd4a4920e3ea7d6d816c6c0b62081fcda34e9118ee0d14b5e Sep 10 22:22:57.215926 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:57.215848 2577 generic.go:356] "Generic (PLEG): container finished" podID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerID="8625d88d7209df80fc0d18cd7d820e15f19c10f354127857e6cd8b5809b8cb8e" exitCode=0 Sep 10 22:22:57.216068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:57.215937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" event={"ID":"aac9e13d-adb9-4a90-991c-1453766a0a98","Type":"ContainerDied","Data":"8625d88d7209df80fc0d18cd7d820e15f19c10f354127857e6cd8b5809b8cb8e"} Sep 10 22:22:57.217451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:57.217429 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" event={"ID":"ae81311b-cd92-434b-be58-71192a85ab62","Type":"ContainerStarted","Data":"53898a741ddbcebdd4a4920e3ea7d6d816c6c0b62081fcda34e9118ee0d14b5e"} Sep 10 22:22:58.226576 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:58.226542 2577 generic.go:356] "Generic (PLEG): container finished" podID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerID="a72f289834247e1e9e2551d319f8c7b23111aa7b0c767c78032e166b47c7bf20" exitCode=0 Sep 10 22:22:58.227068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:58.226606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" event={"ID":"aac9e13d-adb9-4a90-991c-1453766a0a98","Type":"ContainerDied","Data":"a72f289834247e1e9e2551d319f8c7b23111aa7b0c767c78032e166b47c7bf20"} Sep 10 22:22:59.910595 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:59.910571 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:22:59.978850 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:59.978801 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lghd2\" (UniqueName: \"kubernetes.io/projected/aac9e13d-adb9-4a90-991c-1453766a0a98-kube-api-access-lghd2\") pod \"aac9e13d-adb9-4a90-991c-1453766a0a98\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " Sep 10 22:22:59.978967 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:59.978936 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-util\") pod \"aac9e13d-adb9-4a90-991c-1453766a0a98\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " Sep 10 22:22:59.979025 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:59.979012 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-bundle\") pod \"aac9e13d-adb9-4a90-991c-1453766a0a98\" (UID: \"aac9e13d-adb9-4a90-991c-1453766a0a98\") " Sep 10 22:22:59.980034 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:59.979992 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-bundle" (OuterVolumeSpecName: "bundle") pod "aac9e13d-adb9-4a90-991c-1453766a0a98" (UID: "aac9e13d-adb9-4a90-991c-1453766a0a98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:22:59.980934 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:59.980904 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aac9e13d-adb9-4a90-991c-1453766a0a98-kube-api-access-lghd2" (OuterVolumeSpecName: "kube-api-access-lghd2") pod "aac9e13d-adb9-4a90-991c-1453766a0a98" (UID: "aac9e13d-adb9-4a90-991c-1453766a0a98"). InnerVolumeSpecName "kube-api-access-lghd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:22:59.985642 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:22:59.985560 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-util" (OuterVolumeSpecName: "util") pod "aac9e13d-adb9-4a90-991c-1453766a0a98" (UID: "aac9e13d-adb9-4a90-991c-1453766a0a98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:23:00.080307 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:00.080275 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lghd2\" (UniqueName: \"kubernetes.io/projected/aac9e13d-adb9-4a90-991c-1453766a0a98-kube-api-access-lghd2\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:00.080307 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:00.080309 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-util\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:00.080441 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:00.080319 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aac9e13d-adb9-4a90-991c-1453766a0a98-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:00.239197 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:00.239137 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" Sep 10 22:23:00.239197 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:00.239143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0k4zp9" event={"ID":"aac9e13d-adb9-4a90-991c-1453766a0a98","Type":"ContainerDied","Data":"6e103c94a7f669ff4f91d3277c2a492e7b2b515049954b924ec194dd0dd98ef7"} Sep 10 22:23:00.239197 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:00.239182 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e103c94a7f669ff4f91d3277c2a492e7b2b515049954b924ec194dd0dd98ef7" Sep 10 22:23:00.240888 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:00.240863 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" event={"ID":"ae81311b-cd92-434b-be58-71192a85ab62","Type":"ContainerStarted","Data":"bb4903103b4bd4e2b5850dc92de233283418c4e51f1984c43b4c919c5d6f944c"} Sep 10 22:23:01.247332 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:01.247253 2577 generic.go:356] "Generic (PLEG): container finished" podID="ae81311b-cd92-434b-be58-71192a85ab62" containerID="bb4903103b4bd4e2b5850dc92de233283418c4e51f1984c43b4c919c5d6f944c" exitCode=0 Sep 10 22:23:01.247701 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:01.247341 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" event={"ID":"ae81311b-cd92-434b-be58-71192a85ab62","Type":"ContainerDied","Data":"bb4903103b4bd4e2b5850dc92de233283418c4e51f1984c43b4c919c5d6f944c"} Sep 10 22:23:02.254328 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:02.254283 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" event={"ID":"ae81311b-cd92-434b-be58-71192a85ab62","Type":"ContainerStarted","Data":"9fa2b027df43a563199d95e6be6f4af46f439972fce43dbd374285c60bf06556"} Sep 10 22:23:02.254796 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:02.254398 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:23:02.275676 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:02.275618 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" podStartSLOduration=1.9254239480000002 podStartE2EDuration="6.275603905s" totalImagesPullingTime="3.345873507s" totalInitContainerRuntime="1.00430645s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:22:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:02.274867923 +0000 UTC m=+289.071083403" watchObservedRunningTime="2026-09-10 22:23:02.275603905 +0000 UTC m=+289.071819381" Sep 10 22:23:13.261269 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:13.261234 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c969df88c-vr5ft" Sep 10 22:23:13.612610 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:13.612576 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:23:13.615026 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:13.615005 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:23:13.623841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:13.623799 2577 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 22:23:15.251032 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.250994 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w"] Sep 10 22:23:15.253554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.251351 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerName="extract" Sep 10 22:23:15.253554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.251363 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerName="extract" Sep 10 22:23:15.253554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.251379 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerName="pull" Sep 10 22:23:15.253554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.251385 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerName="pull" Sep 10 22:23:15.253554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.251393 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerName="util" Sep 10 22:23:15.253554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.251399 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerName="util" Sep 10 22:23:15.253554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.251469 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aac9e13d-adb9-4a90-991c-1453766a0a98" containerName="extract" Sep 10 22:23:15.254340 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.254325 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.256943 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.256923 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7rppj\"" Sep 10 22:23:15.257569 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.257551 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 10 22:23:15.257709 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.257573 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 10 22:23:15.270494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.270466 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w"] Sep 10 22:23:15.402082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.402030 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98a750a4-0cc1-4692-b5eb-9686a9c7634b-tls-certs\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.402264 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.402169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a750a4-0cc1-4692-b5eb-9686a9c7634b-tmp\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.402264 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.402228 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc8lm\" (UniqueName: \"kubernetes.io/projected/98a750a4-0cc1-4692-b5eb-9686a9c7634b-kube-api-access-bc8lm\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.503587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.503516 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc8lm\" (UniqueName: \"kubernetes.io/projected/98a750a4-0cc1-4692-b5eb-9686a9c7634b-kube-api-access-bc8lm\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.503587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.503553 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98a750a4-0cc1-4692-b5eb-9686a9c7634b-tls-certs\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.503791 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.503607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a750a4-0cc1-4692-b5eb-9686a9c7634b-tmp\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.506051 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.506030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98a750a4-0cc1-4692-b5eb-9686a9c7634b-tmp\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.506192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.506173 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98a750a4-0cc1-4692-b5eb-9686a9c7634b-tls-certs\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.511778 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.511753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc8lm\" (UniqueName: \"kubernetes.io/projected/98a750a4-0cc1-4692-b5eb-9686a9c7634b-kube-api-access-bc8lm\") pod \"kube-auth-proxy-54fc6bbcb5-dpr5w\" (UID: \"98a750a4-0cc1-4692-b5eb-9686a9c7634b\") " pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.564485 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.564463 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" Sep 10 22:23:15.844796 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.844769 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w"] Sep 10 22:23:15.846950 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:23:15.846921 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a750a4_0cc1_4692_b5eb_9686a9c7634b.slice/crio-e8626dacf73f237f16625464108aec127ef4378706de0888035c2681488b426e WatchSource:0}: Error finding container e8626dacf73f237f16625464108aec127ef4378706de0888035c2681488b426e: Status 404 returned error can't find the container with id e8626dacf73f237f16625464108aec127ef4378706de0888035c2681488b426e Sep 10 22:23:15.849049 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:15.849033 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:23:16.322669 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.322632 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" event={"ID":"98a750a4-0cc1-4692-b5eb-9686a9c7634b","Type":"ContainerStarted","Data":"e8626dacf73f237f16625464108aec127ef4378706de0888035c2681488b426e"} Sep 10 22:23:16.595086 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.594292 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm"] Sep 10 22:23:16.598240 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.598217 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.601421 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.601385 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 22:23:16.601759 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.601743 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 22:23:16.601990 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.601973 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7248s\"" Sep 10 22:23:16.602181 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.602166 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 22:23:16.604465 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.604306 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 22:23:16.629378 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.629352 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm"] Sep 10 22:23:16.713098 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.713058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.713270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.713116 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.713270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.713158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl2fg\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-kube-api-access-wl2fg\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.713270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.713182 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.713270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.713223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bcbc635-631d-48be-93b8-0847c4394d8e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.713270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.713244 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.713538 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.713278 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.814552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.814516 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.814749 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.814599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bcbc635-631d-48be-93b8-0847c4394d8e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.814749 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.814629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.814749 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.814677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.814942 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.814713 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.815170 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.815139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.815284 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.815224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl2fg\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-kube-api-access-wl2fg\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.821588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.816347 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.823647 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.823618 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.825919 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.825870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.826105 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.826082 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.826292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.826268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.826480 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.826315 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl2fg\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-kube-api-access-wl2fg\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.826927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.826903 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bcbc635-631d-48be-93b8-0847c4394d8e-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v6zcm\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:16.912127 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:16.912050 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:17.209361 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:17.209315 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm"] Sep 10 22:23:17.211601 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:23:17.211570 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bcbc635_631d_48be_93b8_0847c4394d8e.slice/crio-e4f2a9fd541155b846101062a3e23c42d7e57a4d3a46b9ce82034373e5c7e6c5 WatchSource:0}: Error finding container e4f2a9fd541155b846101062a3e23c42d7e57a4d3a46b9ce82034373e5c7e6c5: Status 404 returned error can't find the container with id e4f2a9fd541155b846101062a3e23c42d7e57a4d3a46b9ce82034373e5c7e6c5 Sep 10 22:23:17.331063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:17.330979 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" event={"ID":"3bcbc635-631d-48be-93b8-0847c4394d8e","Type":"ContainerStarted","Data":"e4f2a9fd541155b846101062a3e23c42d7e57a4d3a46b9ce82034373e5c7e6c5"} Sep 10 22:23:19.344082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:19.344037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" event={"ID":"98a750a4-0cc1-4692-b5eb-9686a9c7634b","Type":"ContainerStarted","Data":"28ad52c5762bd017bca9a2bd8693e5298881a02bbe2afcc7ad64fe3846b9d284"} Sep 10 22:23:19.360981 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:19.360921 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-54fc6bbcb5-dpr5w" podStartSLOduration=1.440887073 podStartE2EDuration="4.360901223s" totalImagesPullingTime="2.92001415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:19.359958382 +0000 UTC m=+306.156173861" watchObservedRunningTime="2026-09-10 22:23:19.360901223 +0000 UTC m=+306.157116701" Sep 10 22:23:20.676099 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:20.676059 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:23:20.676379 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:20.676128 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:23:21.356898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:21.356858 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" event={"ID":"3bcbc635-631d-48be-93b8-0847c4394d8e","Type":"ContainerStarted","Data":"78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176"} Sep 10 22:23:21.357145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:21.357128 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:21.358742 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:21.358717 2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-v6zcm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 22:23:21.358886 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:21.358774 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" podUID="3bcbc635-631d-48be-93b8-0847c4394d8e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 22:23:21.378424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:21.378378 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" podStartSLOduration=1.917217987 podStartE2EDuration="5.378362937s" totalImagesPullingTime="3.46114495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:21.376997815 +0000 UTC m=+308.173213293" watchObservedRunningTime="2026-09-10 22:23:21.378362937 +0000 UTC m=+308.174578414" Sep 10 22:23:22.362385 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:22.362357 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:32.410076 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.410042 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb"] Sep 10 22:23:32.418097 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.418073 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.425733 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.425713 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb"] Sep 10 22:23:32.556168 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.556134 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.556352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.556187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.556352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.556261 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.556352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.556314 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q9tz\" (UniqueName: \"kubernetes.io/projected/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-kube-api-access-2q9tz\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.556489 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.556353 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.556489 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.556425 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.556489 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.556453 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.656915 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.656885 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.657037 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.656922 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.657037 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.656963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.657037 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.656997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.657154 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.657115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.657201 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.657155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q9tz\" (UniqueName: \"kubernetes.io/projected/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-kube-api-access-2q9tz\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.657250 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.657206 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.657963 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.657934 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.659556 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.659524 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.659695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.659650 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.659695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.659648 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.659806 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.659730 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.668428 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.668379 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q9tz\" (UniqueName: \"kubernetes.io/projected/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-kube-api-access-2q9tz\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.668428 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.668385 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-njkbb\" (UID: \"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:32.728088 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:32.728069 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:33.009158 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:33.009130 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb"] Sep 10 22:23:33.011414 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:23:33.011385 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa98b9b_ccfe_423c_9bc1_d3bc780e88d2.slice/crio-26b03a0c808d2088aaceaaf669e10533d0bd92624e8e7d47a42c3777da5ccfe5 WatchSource:0}: Error finding container 26b03a0c808d2088aaceaaf669e10533d0bd92624e8e7d47a42c3777da5ccfe5: Status 404 returned error can't find the container with id 26b03a0c808d2088aaceaaf669e10533d0bd92624e8e7d47a42c3777da5ccfe5 Sep 10 22:23:33.014449 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:33.014412 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:23:33.014544 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:33.014490 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:23:33.421291 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:33.421245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" event={"ID":"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2","Type":"ContainerStarted","Data":"bee11f0eaa6b35f0d9520eced86531a68135ebbaed547a1407b9549f6be0b82f"} Sep 10 22:23:33.422039 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:33.421967 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:33.422039 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:33.422041 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" event={"ID":"dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2","Type":"ContainerStarted","Data":"26b03a0c808d2088aaceaaf669e10533d0bd92624e8e7d47a42c3777da5ccfe5"} Sep 10 22:23:33.449400 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:33.449343 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" podStartSLOduration=1.449320637 podStartE2EDuration="1.449320637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:33.445965829 +0000 UTC m=+320.242181336" watchObservedRunningTime="2026-09-10 22:23:33.449320637 +0000 UTC m=+320.245536115" Sep 10 22:23:34.427391 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.427360 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-njkbb" Sep 10 22:23:34.517327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.517291 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm"] Sep 10 22:23:34.517581 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.517554 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" podUID="3bcbc635-631d-48be-93b8-0847c4394d8e" containerName="discovery" containerID="cri-o://78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176" gracePeriod=30 Sep 10 22:23:34.775736 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.775704 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:34.876373 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876340 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-cacerts\") pod \"3bcbc635-631d-48be-93b8-0847c4394d8e\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " Sep 10 22:23:34.876373 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876377 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-token\") pod \"3bcbc635-631d-48be-93b8-0847c4394d8e\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " Sep 10 22:23:34.876578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876413 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl2fg\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-kube-api-access-wl2fg\") pod \"3bcbc635-631d-48be-93b8-0847c4394d8e\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " Sep 10 22:23:34.876578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876440 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-dns-cert\") pod \"3bcbc635-631d-48be-93b8-0847c4394d8e\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " Sep 10 22:23:34.876578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876460 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-ca-configmap\") pod \"3bcbc635-631d-48be-93b8-0847c4394d8e\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " Sep 10 22:23:34.876578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876499 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-kubeconfig\") pod \"3bcbc635-631d-48be-93b8-0847c4394d8e\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " Sep 10 22:23:34.876578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876523 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bcbc635-631d-48be-93b8-0847c4394d8e-local-certs\") pod \"3bcbc635-631d-48be-93b8-0847c4394d8e\" (UID: \"3bcbc635-631d-48be-93b8-0847c4394d8e\") " Sep 10 22:23:34.876944 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.876913 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3bcbc635-631d-48be-93b8-0847c4394d8e" (UID: "3bcbc635-631d-48be-93b8-0847c4394d8e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:23:34.879150 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.879116 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3bcbc635-631d-48be-93b8-0847c4394d8e" (UID: "3bcbc635-631d-48be-93b8-0847c4394d8e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:23:34.879150 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.879129 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bcbc635-631d-48be-93b8-0847c4394d8e-local-certs" (OuterVolumeSpecName: "local-certs") pod "3bcbc635-631d-48be-93b8-0847c4394d8e" (UID: "3bcbc635-631d-48be-93b8-0847c4394d8e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:23:34.879319 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.879242 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3bcbc635-631d-48be-93b8-0847c4394d8e" (UID: "3bcbc635-631d-48be-93b8-0847c4394d8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:23:34.879319 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.879259 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3bcbc635-631d-48be-93b8-0847c4394d8e" (UID: "3bcbc635-631d-48be-93b8-0847c4394d8e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:23:34.879319 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.879277 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-kube-api-access-wl2fg" (OuterVolumeSpecName: "kube-api-access-wl2fg") pod "3bcbc635-631d-48be-93b8-0847c4394d8e" (UID: "3bcbc635-631d-48be-93b8-0847c4394d8e"). InnerVolumeSpecName "kube-api-access-wl2fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:23:34.879426 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.879324 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-cacerts" (OuterVolumeSpecName: "cacerts") pod "3bcbc635-631d-48be-93b8-0847c4394d8e" (UID: "3bcbc635-631d-48be-93b8-0847c4394d8e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:23:34.977692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.977633 2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-kubeconfig\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:34.977692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.977657 2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bcbc635-631d-48be-93b8-0847c4394d8e-local-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:34.977692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.977666 2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-cacerts\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:34.977692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.977674 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:34.977692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.977682 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl2fg\" (UniqueName: \"kubernetes.io/projected/3bcbc635-631d-48be-93b8-0847c4394d8e-kube-api-access-wl2fg\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:34.977692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.977691 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-dns-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:34.978066 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:34.977699 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3bcbc635-631d-48be-93b8-0847c4394d8e-istio-csr-ca-configmap\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:23:35.431437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.431401 2577 generic.go:356] "Generic (PLEG): container finished" podID="3bcbc635-631d-48be-93b8-0847c4394d8e" containerID="78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176" exitCode=0 Sep 10 22:23:35.431806 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.431449 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" Sep 10 22:23:35.431806 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.431499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" event={"ID":"3bcbc635-631d-48be-93b8-0847c4394d8e","Type":"ContainerDied","Data":"78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176"} Sep 10 22:23:35.431806 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.431555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm" event={"ID":"3bcbc635-631d-48be-93b8-0847c4394d8e","Type":"ContainerDied","Data":"e4f2a9fd541155b846101062a3e23c42d7e57a4d3a46b9ce82034373e5c7e6c5"} Sep 10 22:23:35.431806 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.431578 2577 scope.go:117] "RemoveContainer" containerID="78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176" Sep 10 22:23:35.440908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.440875 2577 scope.go:117] "RemoveContainer" containerID="78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176" Sep 10 22:23:35.441161 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:23:35.441141 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176\": container with ID starting with 78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176 not found: ID does not exist" containerID="78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176" Sep 10 22:23:35.441226 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.441173 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176"} err="failed to get container status \"78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176\": rpc error: code = NotFound desc = could not find container \"78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176\": container with ID starting with 78d87c05fb1ba059aa349b03cfa30061411981c60a631db6c8d3e771907b2176 not found: ID does not exist" Sep 10 22:23:35.476569 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.476546 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm"] Sep 10 22:23:35.480342 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.480319 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v6zcm"] Sep 10 22:23:35.689514 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.689442 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5"] Sep 10 22:23:35.689867 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.689853 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bcbc635-631d-48be-93b8-0847c4394d8e" containerName="discovery" Sep 10 22:23:35.689919 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.689869 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bcbc635-631d-48be-93b8-0847c4394d8e" containerName="discovery" Sep 10 22:23:35.689959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.689943 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bcbc635-631d-48be-93b8-0847c4394d8e" containerName="discovery" Sep 10 22:23:35.692770 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.692750 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:35.695917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.695898 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-p8bf2\"" Sep 10 22:23:35.696015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.695990 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 10 22:23:35.713329 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.713307 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5"] Sep 10 22:23:35.749176 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.749146 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bcbc635-631d-48be-93b8-0847c4394d8e" path="/var/lib/kubelet/pods/3bcbc635-631d-48be-93b8-0847c4394d8e/volumes" Sep 10 22:23:35.786612 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.786581 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18ac3157-48f5-4ec2-897a-68b581b9e537-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7cng5\" (UID: \"18ac3157-48f5-4ec2-897a-68b581b9e537\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:35.786731 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.786616 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl6mm\" (UniqueName: \"kubernetes.io/projected/18ac3157-48f5-4ec2-897a-68b581b9e537-kube-api-access-nl6mm\") pod \"llmisvc-controller-manager-777c6bd7d5-7cng5\" (UID: \"18ac3157-48f5-4ec2-897a-68b581b9e537\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:35.887356 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.887310 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18ac3157-48f5-4ec2-897a-68b581b9e537-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7cng5\" (UID: \"18ac3157-48f5-4ec2-897a-68b581b9e537\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:35.887356 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.887361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl6mm\" (UniqueName: \"kubernetes.io/projected/18ac3157-48f5-4ec2-897a-68b581b9e537-kube-api-access-nl6mm\") pod \"llmisvc-controller-manager-777c6bd7d5-7cng5\" (UID: \"18ac3157-48f5-4ec2-897a-68b581b9e537\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:35.889929 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.889903 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18ac3157-48f5-4ec2-897a-68b581b9e537-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7cng5\" (UID: \"18ac3157-48f5-4ec2-897a-68b581b9e537\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:35.904871 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:35.904842 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl6mm\" (UniqueName: \"kubernetes.io/projected/18ac3157-48f5-4ec2-897a-68b581b9e537-kube-api-access-nl6mm\") pod \"llmisvc-controller-manager-777c6bd7d5-7cng5\" (UID: \"18ac3157-48f5-4ec2-897a-68b581b9e537\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:36.004035 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:36.003962 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:36.273115 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:36.273085 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5"] Sep 10 22:23:36.277043 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:23:36.277012 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod18ac3157_48f5_4ec2_897a_68b581b9e537.slice/crio-6a255366f8044c6ab3b2570fbd6b485271e325e88503aea443692ec659efbfbe WatchSource:0}: Error finding container 6a255366f8044c6ab3b2570fbd6b485271e325e88503aea443692ec659efbfbe: Status 404 returned error can't find the container with id 6a255366f8044c6ab3b2570fbd6b485271e325e88503aea443692ec659efbfbe Sep 10 22:23:36.438647 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:36.438610 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" event={"ID":"18ac3157-48f5-4ec2-897a-68b581b9e537","Type":"ContainerStarted","Data":"6a255366f8044c6ab3b2570fbd6b485271e325e88503aea443692ec659efbfbe"} Sep 10 22:23:39.456443 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:39.456356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" event={"ID":"18ac3157-48f5-4ec2-897a-68b581b9e537","Type":"ContainerStarted","Data":"fe061fec65a46a740a9971bd7b78d33700b147061fbbd05715926bca392254de"} Sep 10 22:23:39.456774 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:39.456460 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:23:39.474051 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:23:39.473979 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" podStartSLOduration=1.568498618 podStartE2EDuration="4.473961747s" totalImagesPullingTime="2.905463129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:39.47378322 +0000 UTC m=+326.269998700" watchObservedRunningTime="2026-09-10 22:23:39.473961747 +0000 UTC m=+326.270177225" Sep 10 22:24:10.465230 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:10.465141 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7cng5" Sep 10 22:24:30.622878 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.622846 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96"] Sep 10 22:24:30.630736 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.630712 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.633247 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.633221 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 10 22:24:30.633393 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.633255 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pmzck\"" Sep 10 22:24:30.633525 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.633502 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 10 22:24:30.634847 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.634797 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96"] Sep 10 22:24:30.636004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.635982 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.636088 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.636028 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.636088 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.636050 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67s2x\" (UniqueName: \"kubernetes.io/projected/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-kube-api-access-67s2x\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.737233 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.737200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.737424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.737261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.737424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.737286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67s2x\" (UniqueName: \"kubernetes.io/projected/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-kube-api-access-67s2x\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.737656 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.737635 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.737724 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.737649 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.739051 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.739032 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx"] Sep 10 22:24:30.743367 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.743349 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:30.745157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.745135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67s2x\" (UniqueName: \"kubernetes.io/projected/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-kube-api-access-67s2x\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:30.749543 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.749522 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx"] Sep 10 22:24:30.821061 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.821030 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq"] Sep 10 22:24:30.825651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.825634 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:30.832051 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.832028 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq"] Sep 10 22:24:30.924221 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.924151 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c"] Sep 10 22:24:30.928549 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.928529 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:30.935629 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.935423 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c"] Sep 10 22:24:30.938327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.938306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td4fd\" (UniqueName: \"kubernetes.io/projected/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-kube-api-access-td4fd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:30.938451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.938347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:30.938451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.938392 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:30.938451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.938421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm4t7\" (UniqueName: \"kubernetes.io/projected/cae5fc7c-ab95-44b1-8d44-c040447e24e2-kube-api-access-rm4t7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:30.938451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.938437 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:30.938649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.938487 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:30.941956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:30.941935 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:31.039545 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td4fd\" (UniqueName: \"kubernetes.io/projected/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-kube-api-access-td4fd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:31.039720 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039568 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:31.039720 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:31.039867 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4zfk\" (UniqueName: \"kubernetes.io/projected/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-kube-api-access-f4zfk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.039867 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.039964 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm4t7\" (UniqueName: \"kubernetes.io/projected/cae5fc7c-ab95-44b1-8d44-c040447e24e2-kube-api-access-rm4t7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:31.039964 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039905 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:31.039964 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.039950 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:31.040134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.040001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.040134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.040017 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:31.040134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.040053 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:31.040252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.040234 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:31.040417 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.040396 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:31.047794 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.047768 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td4fd\" (UniqueName: \"kubernetes.io/projected/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-kube-api-access-td4fd\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:31.047991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.047969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm4t7\" (UniqueName: \"kubernetes.io/projected/cae5fc7c-ab95-44b1-8d44-c040447e24e2-kube-api-access-rm4t7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:31.064752 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.064729 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:31.135199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.135175 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:31.140493 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.140472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4zfk\" (UniqueName: \"kubernetes.io/projected/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-kube-api-access-f4zfk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.140573 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.140505 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.140573 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.140550 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.140936 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.140917 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.140999 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.140936 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.150404 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.150368 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4zfk\" (UniqueName: \"kubernetes.io/projected/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-kube-api-access-f4zfk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.219725 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.219685 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96"] Sep 10 22:24:31.220780 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:24:31.220755 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcac466a1_3cda_4be8_85ae_bb25e8ff70a1.slice/crio-4b134ebf06f9f6a0055b78fda62405dc2281acd7180a804e30b79237ff1488c3 WatchSource:0}: Error finding container 4b134ebf06f9f6a0055b78fda62405dc2281acd7180a804e30b79237ff1488c3: Status 404 returned error can't find the container with id 4b134ebf06f9f6a0055b78fda62405dc2281acd7180a804e30b79237ff1488c3 Sep 10 22:24:31.238116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.238097 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:31.342655 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.342631 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx"] Sep 10 22:24:31.364622 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:24:31.364593 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae5fc7c_ab95_44b1_8d44_c040447e24e2.slice/crio-9ffb5e40ed291eb36ae84b1032a86c57ec37e459fd5347e5d9c02b22da1ddf86 WatchSource:0}: Error finding container 9ffb5e40ed291eb36ae84b1032a86c57ec37e459fd5347e5d9c02b22da1ddf86: Status 404 returned error can't find the container with id 9ffb5e40ed291eb36ae84b1032a86c57ec37e459fd5347e5d9c02b22da1ddf86 Sep 10 22:24:31.409281 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.409254 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq"] Sep 10 22:24:31.413799 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:24:31.413771 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029a2a10_6ba8_4dd1_8ccd_7d4cde4e35ee.slice/crio-0b243ae679754a9a49e210d8047c2d4cfea39bd902f4d65ab713d1c533518ec6 WatchSource:0}: Error finding container 0b243ae679754a9a49e210d8047c2d4cfea39bd902f4d65ab713d1c533518ec6: Status 404 returned error can't find the container with id 0b243ae679754a9a49e210d8047c2d4cfea39bd902f4d65ab713d1c533518ec6 Sep 10 22:24:31.509665 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.509635 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c"] Sep 10 22:24:31.571538 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:24:31.571511 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda979ac2_df0d_4bc4_8a14_cb47f3c55ac7.slice/crio-2df223fe63afba450b2a92599637a774da86b6a3905b24ce1088539ae2bec354 WatchSource:0}: Error finding container 2df223fe63afba450b2a92599637a774da86b6a3905b24ce1088539ae2bec354: Status 404 returned error can't find the container with id 2df223fe63afba450b2a92599637a774da86b6a3905b24ce1088539ae2bec354 Sep 10 22:24:31.719854 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.719768 2577 generic.go:356] "Generic (PLEG): container finished" podID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerID="99b26ea1cf122e33b89b2146cbb982a047bcbc2b8f13f1c3209cc318ac8e3df4" exitCode=0 Sep 10 22:24:31.720194 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.719864 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" event={"ID":"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee","Type":"ContainerDied","Data":"99b26ea1cf122e33b89b2146cbb982a047bcbc2b8f13f1c3209cc318ac8e3df4"} Sep 10 22:24:31.720194 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.719918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" event={"ID":"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee","Type":"ContainerStarted","Data":"0b243ae679754a9a49e210d8047c2d4cfea39bd902f4d65ab713d1c533518ec6"} Sep 10 22:24:31.721690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.721671 2577 generic.go:356] "Generic (PLEG): container finished" podID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerID="4ac12e91689d22d2d3fcae3520a7fbc94f9562457ad1f90f1dbabde0cf447166" exitCode=0 Sep 10 22:24:31.721779 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.721697 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" event={"ID":"cac466a1-3cda-4be8-85ae-bb25e8ff70a1","Type":"ContainerDied","Data":"4ac12e91689d22d2d3fcae3520a7fbc94f9562457ad1f90f1dbabde0cf447166"} Sep 10 22:24:31.721779 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.721743 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" event={"ID":"cac466a1-3cda-4be8-85ae-bb25e8ff70a1","Type":"ContainerStarted","Data":"4b134ebf06f9f6a0055b78fda62405dc2281acd7180a804e30b79237ff1488c3"} Sep 10 22:24:31.723489 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.723468 2577 generic.go:356] "Generic (PLEG): container finished" podID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerID="cd597ba4a9d33bbd7ca732ebd4ef5ff86f73e38ac4d37bb38368f72f110efe2d" exitCode=0 Sep 10 22:24:31.723686 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.723534 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" event={"ID":"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7","Type":"ContainerDied","Data":"cd597ba4a9d33bbd7ca732ebd4ef5ff86f73e38ac4d37bb38368f72f110efe2d"} Sep 10 22:24:31.723686 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.723566 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" event={"ID":"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7","Type":"ContainerStarted","Data":"2df223fe63afba450b2a92599637a774da86b6a3905b24ce1088539ae2bec354"} Sep 10 22:24:31.725163 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.725141 2577 generic.go:356] "Generic (PLEG): container finished" podID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerID="115f1364d966ff84a2804f23c59697bbf0a6fc91b562c4859dd019f57b980370" exitCode=0 Sep 10 22:24:31.725240 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.725218 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" event={"ID":"cae5fc7c-ab95-44b1-8d44-c040447e24e2","Type":"ContainerDied","Data":"115f1364d966ff84a2804f23c59697bbf0a6fc91b562c4859dd019f57b980370"} Sep 10 22:24:31.725298 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:31.725255 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" event={"ID":"cae5fc7c-ab95-44b1-8d44-c040447e24e2","Type":"ContainerStarted","Data":"9ffb5e40ed291eb36ae84b1032a86c57ec37e459fd5347e5d9c02b22da1ddf86"} Sep 10 22:24:33.737973 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.737944 2577 generic.go:356] "Generic (PLEG): container finished" podID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerID="865afa7e097c1811720e13ff4e7d43c1ee4fd1fe6a9d4316156af2093c6a71ab" exitCode=0 Sep 10 22:24:33.738401 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.738026 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" event={"ID":"cac466a1-3cda-4be8-85ae-bb25e8ff70a1","Type":"ContainerDied","Data":"865afa7e097c1811720e13ff4e7d43c1ee4fd1fe6a9d4316156af2093c6a71ab"} Sep 10 22:24:33.740083 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.740057 2577 generic.go:356] "Generic (PLEG): container finished" podID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerID="45b56b47a39d53bf16550b3c3480f46f7dfa72224a885f6bbafe7c08f64f889f" exitCode=0 Sep 10 22:24:33.740330 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.740134 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" event={"ID":"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7","Type":"ContainerDied","Data":"45b56b47a39d53bf16550b3c3480f46f7dfa72224a885f6bbafe7c08f64f889f"} Sep 10 22:24:33.742072 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.742050 2577 generic.go:356] "Generic (PLEG): container finished" podID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerID="58e1e1cc9b27133476e6e5c0e7a8c9031ad8c49617c2d7646add80070f24b98f" exitCode=0 Sep 10 22:24:33.742139 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.742124 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" event={"ID":"cae5fc7c-ab95-44b1-8d44-c040447e24e2","Type":"ContainerDied","Data":"58e1e1cc9b27133476e6e5c0e7a8c9031ad8c49617c2d7646add80070f24b98f"} Sep 10 22:24:33.744269 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.744248 2577 generic.go:356] "Generic (PLEG): container finished" podID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerID="0594ac9e0b0b5ff36d7016fbf26ed9db1575b5b79372cb2a8057f3e88f1b7ccb" exitCode=0 Sep 10 22:24:33.752895 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:33.752686 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" event={"ID":"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee","Type":"ContainerDied","Data":"0594ac9e0b0b5ff36d7016fbf26ed9db1575b5b79372cb2a8057f3e88f1b7ccb"} Sep 10 22:24:34.751834 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.751775 2577 generic.go:356] "Generic (PLEG): container finished" podID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerID="e85c3de4af770340fd61bfbf8ff1c4c25ccbeed685e9beb45d33fcf6de2da8b6" exitCode=0 Sep 10 22:24:34.752234 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.751864 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" event={"ID":"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7","Type":"ContainerDied","Data":"e85c3de4af770340fd61bfbf8ff1c4c25ccbeed685e9beb45d33fcf6de2da8b6"} Sep 10 22:24:34.754004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.753984 2577 generic.go:356] "Generic (PLEG): container finished" podID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerID="420359700be021ad3ba7ee3ac1dde8aa57ebf71cc9d98de85a096d0a13077681" exitCode=0 Sep 10 22:24:34.754121 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.754058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" event={"ID":"cae5fc7c-ab95-44b1-8d44-c040447e24e2","Type":"ContainerDied","Data":"420359700be021ad3ba7ee3ac1dde8aa57ebf71cc9d98de85a096d0a13077681"} Sep 10 22:24:34.756266 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.756247 2577 generic.go:356] "Generic (PLEG): container finished" podID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerID="4124e9d516d426a52b6f74c850f0bad7bb0661bfef7725b0894028ecd2b70436" exitCode=0 Sep 10 22:24:34.756375 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.756324 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" event={"ID":"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee","Type":"ContainerDied","Data":"4124e9d516d426a52b6f74c850f0bad7bb0661bfef7725b0894028ecd2b70436"} Sep 10 22:24:34.758401 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.758381 2577 generic.go:356] "Generic (PLEG): container finished" podID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerID="85550b4b4bac5c713d20bfa6f7ee9b72862e2e2f2861007e6f6992affcd6b0b5" exitCode=0 Sep 10 22:24:34.758515 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:34.758406 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" event={"ID":"cac466a1-3cda-4be8-85ae-bb25e8ff70a1","Type":"ContainerDied","Data":"85550b4b4bac5c713d20bfa6f7ee9b72862e2e2f2861007e6f6992affcd6b0b5"} Sep 10 22:24:35.931748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.931592 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:35.934690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.934666 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:35.956982 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.956956 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:35.965793 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.965770 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:35.982516 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982493 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm4t7\" (UniqueName: \"kubernetes.io/projected/cae5fc7c-ab95-44b1-8d44-c040447e24e2-kube-api-access-rm4t7\") pod \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " Sep 10 22:24:35.982631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982535 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-util\") pod \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " Sep 10 22:24:35.982631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982592 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67s2x\" (UniqueName: \"kubernetes.io/projected/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-kube-api-access-67s2x\") pod \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " Sep 10 22:24:35.982631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982617 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-util\") pod \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " Sep 10 22:24:35.982802 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982645 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-bundle\") pod \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " Sep 10 22:24:35.982802 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982671 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-bundle\") pod \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " Sep 10 22:24:35.982802 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982697 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-bundle\") pod \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\" (UID: \"cae5fc7c-ab95-44b1-8d44-c040447e24e2\") " Sep 10 22:24:35.982802 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982742 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-bundle\") pod \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " Sep 10 22:24:35.982802 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982768 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4zfk\" (UniqueName: \"kubernetes.io/projected/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-kube-api-access-f4zfk\") pod \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " Sep 10 22:24:35.983108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982802 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-util\") pod \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\" (UID: \"cac466a1-3cda-4be8-85ae-bb25e8ff70a1\") " Sep 10 22:24:35.983108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982865 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-util\") pod \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\" (UID: \"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7\") " Sep 10 22:24:35.983108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.982909 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td4fd\" (UniqueName: \"kubernetes.io/projected/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-kube-api-access-td4fd\") pod \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\" (UID: \"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee\") " Sep 10 22:24:35.985918 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.985342 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-bundle" (OuterVolumeSpecName: "bundle") pod "cae5fc7c-ab95-44b1-8d44-c040447e24e2" (UID: "cae5fc7c-ab95-44b1-8d44-c040447e24e2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:35.985918 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.985431 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-bundle" (OuterVolumeSpecName: "bundle") pod "cac466a1-3cda-4be8-85ae-bb25e8ff70a1" (UID: "cac466a1-3cda-4be8-85ae-bb25e8ff70a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:35.986100 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.986067 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-kube-api-access-td4fd" (OuterVolumeSpecName: "kube-api-access-td4fd") pod "029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" (UID: "029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee"). InnerVolumeSpecName "kube-api-access-td4fd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:24:35.988965 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.987718 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-bundle" (OuterVolumeSpecName: "bundle") pod "da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" (UID: "da979ac2-df0d-4bc4-8a14-cb47f3c55ac7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:35.988965 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.988157 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-bundle" (OuterVolumeSpecName: "bundle") pod "029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" (UID: "029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:35.992054 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.991401 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae5fc7c-ab95-44b1-8d44-c040447e24e2-kube-api-access-rm4t7" (OuterVolumeSpecName: "kube-api-access-rm4t7") pod "cae5fc7c-ab95-44b1-8d44-c040447e24e2" (UID: "cae5fc7c-ab95-44b1-8d44-c040447e24e2"). InnerVolumeSpecName "kube-api-access-rm4t7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:24:35.992054 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.991467 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-kube-api-access-f4zfk" (OuterVolumeSpecName: "kube-api-access-f4zfk") pod "da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" (UID: "da979ac2-df0d-4bc4-8a14-cb47f3c55ac7"). InnerVolumeSpecName "kube-api-access-f4zfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:24:35.992506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.992478 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-util" (OuterVolumeSpecName: "util") pod "cac466a1-3cda-4be8-85ae-bb25e8ff70a1" (UID: "cac466a1-3cda-4be8-85ae-bb25e8ff70a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:35.992617 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.992572 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-kube-api-access-67s2x" (OuterVolumeSpecName: "kube-api-access-67s2x") pod "cac466a1-3cda-4be8-85ae-bb25e8ff70a1" (UID: "cac466a1-3cda-4be8-85ae-bb25e8ff70a1"). InnerVolumeSpecName "kube-api-access-67s2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:24:35.993257 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.993228 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-util" (OuterVolumeSpecName: "util") pod "029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" (UID: "029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:35.993354 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.993337 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-util" (OuterVolumeSpecName: "util") pod "da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" (UID: "da979ac2-df0d-4bc4-8a14-cb47f3c55ac7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:35.996286 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:35.996263 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-util" (OuterVolumeSpecName: "util") pod "cae5fc7c-ab95-44b1-8d44-c040447e24e2" (UID: "cae5fc7c-ab95-44b1-8d44-c040447e24e2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:24:36.083688 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083661 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td4fd\" (UniqueName: \"kubernetes.io/projected/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-kube-api-access-td4fd\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083688 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083684 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm4t7\" (UniqueName: \"kubernetes.io/projected/cae5fc7c-ab95-44b1-8d44-c040447e24e2-kube-api-access-rm4t7\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083699 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-util\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083719 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67s2x\" (UniqueName: \"kubernetes.io/projected/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-kube-api-access-67s2x\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083727 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-util\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083736 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083743 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083751 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cae5fc7c-ab95-44b1-8d44-c040447e24e2-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083759 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee-bundle\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083767 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4zfk\" (UniqueName: \"kubernetes.io/projected/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-kube-api-access-f4zfk\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083781 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cac466a1-3cda-4be8-85ae-bb25e8ff70a1-util\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.083898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.083790 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da979ac2-df0d-4bc4-8a14-cb47f3c55ac7-util\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:24:36.769518 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.769487 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" Sep 10 22:24:36.769518 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.769510 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7x7ntq" event={"ID":"029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee","Type":"ContainerDied","Data":"0b243ae679754a9a49e210d8047c2d4cfea39bd902f4d65ab713d1c533518ec6"} Sep 10 22:24:36.769774 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.769560 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b243ae679754a9a49e210d8047c2d4cfea39bd902f4d65ab713d1c533518ec6" Sep 10 22:24:36.771693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.771669 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" Sep 10 22:24:36.771693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.771678 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ctxx96" event={"ID":"cac466a1-3cda-4be8-85ae-bb25e8ff70a1","Type":"ContainerDied","Data":"4b134ebf06f9f6a0055b78fda62405dc2281acd7180a804e30b79237ff1488c3"} Sep 10 22:24:36.771893 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.771724 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b134ebf06f9f6a0055b78fda62405dc2281acd7180a804e30b79237ff1488c3" Sep 10 22:24:36.773842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.773794 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" event={"ID":"da979ac2-df0d-4bc4-8a14-cb47f3c55ac7","Type":"ContainerDied","Data":"2df223fe63afba450b2a92599637a774da86b6a3905b24ce1088539ae2bec354"} Sep 10 22:24:36.773842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.773832 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azbl9c" Sep 10 22:24:36.774032 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.773845 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2df223fe63afba450b2a92599637a774da86b6a3905b24ce1088539ae2bec354" Sep 10 22:24:36.776057 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.776034 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" event={"ID":"cae5fc7c-ab95-44b1-8d44-c040447e24e2","Type":"ContainerDied","Data":"9ffb5e40ed291eb36ae84b1032a86c57ec37e459fd5347e5d9c02b22da1ddf86"} Sep 10 22:24:36.776136 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.776073 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ffb5e40ed291eb36ae84b1032a86c57ec37e459fd5347e5d9c02b22da1ddf86" Sep 10 22:24:36.776136 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:36.776041 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62nklx" Sep 10 22:24:42.328499 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.328457 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dd66f7944-j62zz"] Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.328997 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerName="pull" Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329015 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerName="pull" Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329028 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerName="pull" Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329037 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerName="pull" Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329059 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerName="extract" Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329067 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerName="extract" Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329081 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerName="pull" Sep 10 22:24:42.329095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329090 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerName="pull" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329103 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329111 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329122 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329130 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329138 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329146 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329157 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329164 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329180 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerName="pull" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329188 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerName="pull" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329197 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329205 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329215 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329223 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerName="util" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329232 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329240 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329321 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cac466a1-3cda-4be8-85ae-bb25e8ff70a1" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329336 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="029a2a10-6ba8-4dd1-8ccd-7d4cde4e35ee" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329345 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cae5fc7c-ab95-44b1-8d44-c040447e24e2" containerName="extract" Sep 10 22:24:42.329477 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.329355 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="da979ac2-df0d-4bc4-8a14-cb47f3c55ac7" containerName="extract" Sep 10 22:24:42.334091 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.334070 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.336543 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.336521 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 22:24:42.337123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.336987 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 22:24:42.337123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.337010 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 22:24:42.337123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.336990 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 22:24:42.337123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.337092 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 22:24:42.337123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.336990 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 22:24:42.337468 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.337207 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 22:24:42.337468 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.337264 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zzpsp\"" Sep 10 22:24:42.342566 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.342529 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 22:24:42.345678 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.345655 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dd66f7944-j62zz"] Sep 10 22:24:42.436021 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.435987 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-config\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.436021 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.436026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-service-ca\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.436262 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.436043 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-oauth-serving-cert\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.436262 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.436100 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-trusted-ca-bundle\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.436262 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.436210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-oauth-config\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.436262 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.436250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-serving-cert\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.436459 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.436315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99xxn\" (UniqueName: \"kubernetes.io/projected/0261df3f-a1ae-480d-8e50-413d425fa8f2-kube-api-access-99xxn\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537078 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-service-ca\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537078 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537080 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-oauth-serving-cert\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-trusted-ca-bundle\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-oauth-config\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537201 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-serving-cert\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537257 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99xxn\" (UniqueName: \"kubernetes.io/projected/0261df3f-a1ae-480d-8e50-413d425fa8f2-kube-api-access-99xxn\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537564 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-config\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537958 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537922 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-service-ca\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.537958 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.537922 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-oauth-serving-cert\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.538126 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.538021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-config\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.538229 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.538210 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0261df3f-a1ae-480d-8e50-413d425fa8f2-trusted-ca-bundle\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.540288 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.540273 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-oauth-config\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.540502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.540483 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0261df3f-a1ae-480d-8e50-413d425fa8f2-console-serving-cert\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.545683 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.545655 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99xxn\" (UniqueName: \"kubernetes.io/projected/0261df3f-a1ae-480d-8e50-413d425fa8f2-kube-api-access-99xxn\") pod \"console-7dd66f7944-j62zz\" (UID: \"0261df3f-a1ae-480d-8e50-413d425fa8f2\") " pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.645043 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.644986 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:42.927115 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:42.927091 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dd66f7944-j62zz"] Sep 10 22:24:42.929490 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:24:42.929463 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0261df3f_a1ae_480d_8e50_413d425fa8f2.slice/crio-d4d54c86b831aa2c7a2275f3d555643076dcaf9631508b5a3761949389da7cb6 WatchSource:0}: Error finding container d4d54c86b831aa2c7a2275f3d555643076dcaf9631508b5a3761949389da7cb6: Status 404 returned error can't find the container with id d4d54c86b831aa2c7a2275f3d555643076dcaf9631508b5a3761949389da7cb6 Sep 10 22:24:43.813031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:43.812999 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dd66f7944-j62zz" event={"ID":"0261df3f-a1ae-480d-8e50-413d425fa8f2","Type":"ContainerStarted","Data":"503a8248dc6e8f711556397cb99b3ce6673dcf170a95495e223ec3f8e6bcd195"} Sep 10 22:24:43.813456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:43.813049 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dd66f7944-j62zz" event={"ID":"0261df3f-a1ae-480d-8e50-413d425fa8f2","Type":"ContainerStarted","Data":"d4d54c86b831aa2c7a2275f3d555643076dcaf9631508b5a3761949389da7cb6"} Sep 10 22:24:43.832513 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:43.832465 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dd66f7944-j62zz" podStartSLOduration=1.832450642 podStartE2EDuration="1.832450642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:24:43.831355757 +0000 UTC m=+390.627571236" watchObservedRunningTime="2026-09-10 22:24:43.832450642 +0000 UTC m=+390.628666119" Sep 10 22:24:45.678792 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.678758 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649"] Sep 10 22:24:45.682348 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.682332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:45.687238 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.687217 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 10 22:24:45.687348 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.687286 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 10 22:24:45.687348 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.687300 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qplrh\"" Sep 10 22:24:45.704266 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.704241 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649"] Sep 10 22:24:45.765933 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.765905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csh6t\" (UniqueName: \"kubernetes.io/projected/d4811e8f-3239-4c95-b304-601b832e3268-kube-api-access-csh6t\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:45.766106 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.766029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4811e8f-3239-4c95-b304-601b832e3268-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:45.866376 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.866341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csh6t\" (UniqueName: \"kubernetes.io/projected/d4811e8f-3239-4c95-b304-601b832e3268-kube-api-access-csh6t\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:45.866542 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.866444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4811e8f-3239-4c95-b304-601b832e3268-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:45.866805 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.866784 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4811e8f-3239-4c95-b304-601b832e3268-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:45.881923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.881898 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csh6t\" (UniqueName: \"kubernetes.io/projected/d4811e8f-3239-4c95-b304-601b832e3268-kube-api-access-csh6t\") pod \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:45.993451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:45.993356 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:24:46.295577 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:46.295546 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649"] Sep 10 22:24:46.298167 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:24:46.298125 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4811e8f_3239_4c95_b304_601b832e3268.slice/crio-b04534a0813f4500ba8e32fae5dd86fe4144fb9aedf851f383dbabadf1809df1 WatchSource:0}: Error finding container b04534a0813f4500ba8e32fae5dd86fe4144fb9aedf851f383dbabadf1809df1: Status 404 returned error can't find the container with id b04534a0813f4500ba8e32fae5dd86fe4144fb9aedf851f383dbabadf1809df1 Sep 10 22:24:46.835810 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:46.835773 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" event={"ID":"d4811e8f-3239-4c95-b304-601b832e3268","Type":"ContainerStarted","Data":"b04534a0813f4500ba8e32fae5dd86fe4144fb9aedf851f383dbabadf1809df1"} Sep 10 22:24:50.582166 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.582134 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22"] Sep 10 22:24:50.587141 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.587122 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:24:50.589288 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.589266 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4rmvb\"" Sep 10 22:24:50.599377 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.599355 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22"] Sep 10 22:24:50.708059 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.708028 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttgj5\" (UniqueName: \"kubernetes.io/projected/ce23a3a5-7b92-40a2-a773-76c2cb99d0ad-kube-api-access-ttgj5\") pod \"limitador-operator-controller-manager-74495c69f8-mks22\" (UID: \"ce23a3a5-7b92-40a2-a773-76c2cb99d0ad\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:24:50.808463 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.808436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgj5\" (UniqueName: \"kubernetes.io/projected/ce23a3a5-7b92-40a2-a773-76c2cb99d0ad-kube-api-access-ttgj5\") pod \"limitador-operator-controller-manager-74495c69f8-mks22\" (UID: \"ce23a3a5-7b92-40a2-a773-76c2cb99d0ad\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:24:50.825156 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.825130 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgj5\" (UniqueName: \"kubernetes.io/projected/ce23a3a5-7b92-40a2-a773-76c2cb99d0ad-kube-api-access-ttgj5\") pod \"limitador-operator-controller-manager-74495c69f8-mks22\" (UID: \"ce23a3a5-7b92-40a2-a773-76c2cb99d0ad\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:24:50.897282 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:50.897260 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:24:51.178869 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:51.178841 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22"] Sep 10 22:24:51.180630 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:24:51.180607 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce23a3a5_7b92_40a2_a773_76c2cb99d0ad.slice/crio-63e579146cc87199fb365913b766f834001f66919711e647bc61ca7df8b6952e WatchSource:0}: Error finding container 63e579146cc87199fb365913b766f834001f66919711e647bc61ca7df8b6952e: Status 404 returned error can't find the container with id 63e579146cc87199fb365913b766f834001f66919711e647bc61ca7df8b6952e Sep 10 22:24:51.863634 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:51.863598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" event={"ID":"ce23a3a5-7b92-40a2-a773-76c2cb99d0ad","Type":"ContainerStarted","Data":"63e579146cc87199fb365913b766f834001f66919711e647bc61ca7df8b6952e"} Sep 10 22:24:52.646142 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:52.646101 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:52.646301 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:52.646173 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:52.652137 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:52.652114 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:52.876025 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:52.875953 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dd66f7944-j62zz" Sep 10 22:24:53.876968 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:53.876927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" event={"ID":"ce23a3a5-7b92-40a2-a773-76c2cb99d0ad","Type":"ContainerStarted","Data":"67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c"} Sep 10 22:24:53.877355 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:53.876999 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:24:53.899050 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:24:53.899005 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" podStartSLOduration=1.35570784 podStartE2EDuration="3.898988571s" totalImagesPullingTime="2.543280731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:24:53.897158313 +0000 UTC m=+400.693373801" watchObservedRunningTime="2026-09-10 22:24:53.898988571 +0000 UTC m=+400.695204048" Sep 10 22:25:04.885746 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:04.885706 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:06.954829 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:06.954772 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" event={"ID":"d4811e8f-3239-4c95-b304-601b832e3268","Type":"ContainerStarted","Data":"7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7"} Sep 10 22:25:06.955234 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:06.954867 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:06.975670 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:06.975619 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" podStartSLOduration=2.190685027 podStartE2EDuration="21.97560086s" totalImagesPullingTime="19.784915833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:06.974861572 +0000 UTC m=+413.771077046" watchObservedRunningTime="2026-09-10 22:25:06.97560086 +0000 UTC m=+413.771816338" Sep 10 22:25:17.962484 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:17.962450 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.108484 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.108445 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649"] Sep 10 22:25:20.108909 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.108671 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" podUID="d4811e8f-3239-4c95-b304-601b832e3268" containerName="manager" containerID="cri-o://7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7" gracePeriod=2 Sep 10 22:25:20.126608 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.126575 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649"] Sep 10 22:25:20.133396 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.133365 2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d4811e8f-3239-4c95-b304-601b832e3268\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-zl649\": pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.137673 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.137647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744"] Sep 10 22:25:20.138031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.138018 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4811e8f-3239-4c95-b304-601b832e3268" containerName="manager" Sep 10 22:25:20.138104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.138033 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4811e8f-3239-4c95-b304-601b832e3268" containerName="manager" Sep 10 22:25:20.138161 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.138118 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4811e8f-3239-4c95-b304-601b832e3268" containerName="manager" Sep 10 22:25:20.141269 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.141249 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.145505 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.145475 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.147985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.147960 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22"] Sep 10 22:25:20.148196 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.148152 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" containerName="manager" containerID="cri-o://67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c" gracePeriod=2 Sep 10 22:25:20.161617 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.161580 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22"] Sep 10 22:25:20.165360 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.165336 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744"] Sep 10 22:25:20.169053 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.169029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xld92\" (UniqueName: \"kubernetes.io/projected/fa42d570-0e77-43a1-960f-7d3ee7ec554b-kube-api-access-xld92\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2j744\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.169154 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.169069 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa42d570-0e77-43a1-960f-7d3ee7ec554b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2j744\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.180655 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.180634 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db"] Sep 10 22:25:20.181032 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.181018 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" containerName="manager" Sep 10 22:25:20.181103 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.181034 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" containerName="manager" Sep 10 22:25:20.181149 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.181108 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" containerName="manager" Sep 10 22:25:20.183760 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.183731 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:20.184353 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.184331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" Sep 10 22:25:20.185290 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.185260 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.186725 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.186703 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:20.204411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.204385 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db"] Sep 10 22:25:20.238883 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.238849 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.270498 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.270469 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa42d570-0e77-43a1-960f-7d3ee7ec554b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2j744\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.270654 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.270510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfwnr\" (UniqueName: \"kubernetes.io/projected/c2453d00-64cd-4eae-a126-2cf3428a04d9-kube-api-access-jfwnr\") pod \"limitador-operator-controller-manager-74495c69f8-lq8db\" (UID: \"c2453d00-64cd-4eae-a126-2cf3428a04d9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" Sep 10 22:25:20.270654 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.270606 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xld92\" (UniqueName: \"kubernetes.io/projected/fa42d570-0e77-43a1-960f-7d3ee7ec554b-kube-api-access-xld92\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2j744\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.271037 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.271008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa42d570-0e77-43a1-960f-7d3ee7ec554b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2j744\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.284474 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.284438 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xld92\" (UniqueName: \"kubernetes.io/projected/fa42d570-0e77-43a1-960f-7d3ee7ec554b-kube-api-access-xld92\") pod \"kuadrant-operator-controller-manager-76f8f686bf-2j744\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.371860 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.371828 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwnr\" (UniqueName: \"kubernetes.io/projected/c2453d00-64cd-4eae-a126-2cf3428a04d9-kube-api-access-jfwnr\") pod \"limitador-operator-controller-manager-74495c69f8-lq8db\" (UID: \"c2453d00-64cd-4eae-a126-2cf3428a04d9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" Sep 10 22:25:20.381399 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.381366 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfwnr\" (UniqueName: \"kubernetes.io/projected/c2453d00-64cd-4eae-a126-2cf3428a04d9-kube-api-access-jfwnr\") pod \"limitador-operator-controller-manager-74495c69f8-lq8db\" (UID: \"c2453d00-64cd-4eae-a126-2cf3428a04d9\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" Sep 10 22:25:20.390465 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.390444 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.394272 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.394255 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:20.394640 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.394613 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.396202 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.396181 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:20.398099 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.398078 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:20.399739 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:20.399707 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:20.472478 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.472451 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csh6t\" (UniqueName: \"kubernetes.io/projected/d4811e8f-3239-4c95-b304-601b832e3268-kube-api-access-csh6t\") pod \"d4811e8f-3239-4c95-b304-601b832e3268\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " Sep 10 22:25:20.472631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.472496 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttgj5\" (UniqueName: \"kubernetes.io/projected/ce23a3a5-7b92-40a2-a773-76c2cb99d0ad-kube-api-access-ttgj5\") pod \"ce23a3a5-7b92-40a2-a773-76c2cb99d0ad\" (UID: \"ce23a3a5-7b92-40a2-a773-76c2cb99d0ad\") " Sep 10 22:25:20.472631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.472559 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4811e8f-3239-4c95-b304-601b832e3268-extensions-socket-volume\") pod \"d4811e8f-3239-4c95-b304-601b832e3268\" (UID: \"d4811e8f-3239-4c95-b304-601b832e3268\") " Sep 10 22:25:20.472972 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.472940 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4811e8f-3239-4c95-b304-601b832e3268-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d4811e8f-3239-4c95-b304-601b832e3268" (UID: "d4811e8f-3239-4c95-b304-601b832e3268"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:25:20.474755 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.474735 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4811e8f-3239-4c95-b304-601b832e3268-kube-api-access-csh6t" (OuterVolumeSpecName: "kube-api-access-csh6t") pod "d4811e8f-3239-4c95-b304-601b832e3268" (UID: "d4811e8f-3239-4c95-b304-601b832e3268"). InnerVolumeSpecName "kube-api-access-csh6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:25:20.474860 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.474797 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce23a3a5-7b92-40a2-a773-76c2cb99d0ad-kube-api-access-ttgj5" (OuterVolumeSpecName: "kube-api-access-ttgj5") pod "ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" (UID: "ce23a3a5-7b92-40a2-a773-76c2cb99d0ad"). InnerVolumeSpecName "kube-api-access-ttgj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:25:20.549078 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.549044 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:20.554364 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.554348 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" Sep 10 22:25:20.574146 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.574117 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4811e8f-3239-4c95-b304-601b832e3268-extensions-socket-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:25:20.574237 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.574152 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csh6t\" (UniqueName: \"kubernetes.io/projected/d4811e8f-3239-4c95-b304-601b832e3268-kube-api-access-csh6t\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:25:20.574237 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.574168 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttgj5\" (UniqueName: \"kubernetes.io/projected/ce23a3a5-7b92-40a2-a773-76c2cb99d0ad-kube-api-access-ttgj5\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:25:20.831198 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.831175 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744"] Sep 10 22:25:20.833331 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:20.833303 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db"] Sep 10 22:25:20.833549 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:25:20.833523 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2453d00_64cd_4eae_a126_2cf3428a04d9.slice/crio-177af303ee610a3e6a9203ef29a053ae2a7e8828dedf2b18056ebd9b14b9651c WatchSource:0}: Error finding container 177af303ee610a3e6a9203ef29a053ae2a7e8828dedf2b18056ebd9b14b9651c: Status 404 returned error can't find the container with id 177af303ee610a3e6a9203ef29a053ae2a7e8828dedf2b18056ebd9b14b9651c Sep 10 22:25:20.834325 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:25:20.834238 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa42d570_0e77_43a1_960f_7d3ee7ec554b.slice/crio-16c6fcc9a7e2983c3f2858252e69b6db003f948a3e29a3657bf3d6b9924b0fba WatchSource:0}: Error finding container 16c6fcc9a7e2983c3f2858252e69b6db003f948a3e29a3657bf3d6b9924b0fba: Status 404 returned error can't find the container with id 16c6fcc9a7e2983c3f2858252e69b6db003f948a3e29a3657bf3d6b9924b0fba Sep 10 22:25:21.037576 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.037538 2577 generic.go:356] "Generic (PLEG): container finished" podID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" containerID="67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c" exitCode=0 Sep 10 22:25:21.037758 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.037583 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:21.037758 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.037646 2577 scope.go:117] "RemoveContainer" containerID="67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c" Sep 10 22:25:21.039330 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.039267 2577 generic.go:356] "Generic (PLEG): container finished" podID="d4811e8f-3239-4c95-b304-601b832e3268" containerID="7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7" exitCode=0 Sep 10 22:25:21.039431 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.039369 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:21.039538 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.039516 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:21.041135 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.041097 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:21.041479 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.041448 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" event={"ID":"c2453d00-64cd-4eae-a126-2cf3428a04d9","Type":"ContainerStarted","Data":"cff4af00bbdc83b15f0ee90ed8e35491ef30ea70eeeb802fd73834fb9c0787fa"} Sep 10 22:25:21.041578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.041508 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" Sep 10 22:25:21.041578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.041557 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" event={"ID":"c2453d00-64cd-4eae-a126-2cf3428a04d9","Type":"ContainerStarted","Data":"177af303ee610a3e6a9203ef29a053ae2a7e8828dedf2b18056ebd9b14b9651c"} Sep 10 22:25:21.042724 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.042689 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:21.043782 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.043757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" event={"ID":"fa42d570-0e77-43a1-960f-7d3ee7ec554b","Type":"ContainerStarted","Data":"be434d5730c9ad3de4c0d0fcd36382a34ce478fea0099072af97453cdaf4486b"} Sep 10 22:25:21.043918 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.043805 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" event={"ID":"fa42d570-0e77-43a1-960f-7d3ee7ec554b","Type":"ContainerStarted","Data":"16c6fcc9a7e2983c3f2858252e69b6db003f948a3e29a3657bf3d6b9924b0fba"} Sep 10 22:25:21.043918 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.043860 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:21.044105 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.044083 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:21.045537 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.045513 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:21.048171 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.048156 2577 scope.go:117] "RemoveContainer" containerID="67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c" Sep 10 22:25:21.048451 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.048431 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c\": container with ID starting with 67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c not found: ID does not exist" containerID="67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c" Sep 10 22:25:21.048510 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.048466 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c"} err="failed to get container status \"67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c\": rpc error: code = NotFound desc = could not find container \"67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c\": container with ID starting with 67614a9ba3de29363c2cff417925ee6a039c0d3e083e93aceb2271c67902fa8c not found: ID does not exist" Sep 10 22:25:21.048510 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.048483 2577 scope.go:117] "RemoveContainer" containerID="7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7" Sep 10 22:25:21.056853 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.056834 2577 scope.go:117] "RemoveContainer" containerID="7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7" Sep 10 22:25:21.057122 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.057104 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7\": container with ID starting with 7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7 not found: ID does not exist" containerID="7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7" Sep 10 22:25:21.057167 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.057128 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7"} err="failed to get container status \"7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7\": rpc error: code = NotFound desc = could not find container \"7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7\": container with ID starting with 7eb768b3006ed457e3f8e72c92847c707d040dd64b11d2b35ce88d1b00fb5ec7 not found: ID does not exist" Sep 10 22:25:21.067301 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.067255 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" podStartSLOduration=1.067239646 podStartE2EDuration="1.067239646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:21.065167829 +0000 UTC m=+427.861383308" watchObservedRunningTime="2026-09-10 22:25:21.067239646 +0000 UTC m=+427.863455124" Sep 10 22:25:21.100008 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.099931 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:21.100794 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.100756 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" podStartSLOduration=1.100739268 podStartE2EDuration="1.100739268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:21.098191426 +0000 UTC m=+427.894406930" watchObservedRunningTime="2026-09-10 22:25:21.100739268 +0000 UTC m=+427.896954772" Sep 10 22:25:21.101489 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.101464 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-mks22\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mks22" Sep 10 22:25:21.103148 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:25:21.103114 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-zl649\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="d4811e8f-3239-4c95-b304-601b832e3268" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-zl649" Sep 10 22:25:21.749558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.749522 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce23a3a5-7b92-40a2-a773-76c2cb99d0ad" path="/var/lib/kubelet/pods/ce23a3a5-7b92-40a2-a773-76c2cb99d0ad/volumes" Sep 10 22:25:21.749930 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:21.749894 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4811e8f-3239-4c95-b304-601b832e3268" path="/var/lib/kubelet/pods/d4811e8f-3239-4c95-b304-601b832e3268/volumes" Sep 10 22:25:32.053894 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:32.053862 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-lq8db" Sep 10 22:25:32.054416 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:32.053939 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:35.925571 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:35.925488 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744"] Sep 10 22:25:35.925954 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:35.925735 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" podUID="fa42d570-0e77-43a1-960f-7d3ee7ec554b" containerName="manager" containerID="cri-o://be434d5730c9ad3de4c0d0fcd36382a34ce478fea0099072af97453cdaf4486b" gracePeriod=10 Sep 10 22:25:36.131744 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.131704 2577 generic.go:356] "Generic (PLEG): container finished" podID="fa42d570-0e77-43a1-960f-7d3ee7ec554b" containerID="be434d5730c9ad3de4c0d0fcd36382a34ce478fea0099072af97453cdaf4486b" exitCode=0 Sep 10 22:25:36.131932 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.131774 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" event={"ID":"fa42d570-0e77-43a1-960f-7d3ee7ec554b","Type":"ContainerDied","Data":"be434d5730c9ad3de4c0d0fcd36382a34ce478fea0099072af97453cdaf4486b"} Sep 10 22:25:36.172457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.172433 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:36.185065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.185012 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h"] Sep 10 22:25:36.185456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.185437 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa42d570-0e77-43a1-960f-7d3ee7ec554b" containerName="manager" Sep 10 22:25:36.185456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.185456 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa42d570-0e77-43a1-960f-7d3ee7ec554b" containerName="manager" Sep 10 22:25:36.185624 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.185552 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa42d570-0e77-43a1-960f-7d3ee7ec554b" containerName="manager" Sep 10 22:25:36.187472 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.187458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.210089 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.210065 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h"] Sep 10 22:25:36.307709 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.307679 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xld92\" (UniqueName: \"kubernetes.io/projected/fa42d570-0e77-43a1-960f-7d3ee7ec554b-kube-api-access-xld92\") pod \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " Sep 10 22:25:36.307927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.307749 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa42d570-0e77-43a1-960f-7d3ee7ec554b-extensions-socket-volume\") pod \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\" (UID: \"fa42d570-0e77-43a1-960f-7d3ee7ec554b\") " Sep 10 22:25:36.308010 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.307987 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11358f6d-0980-4257-b4ff-e0c254bdedde-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7tt5h\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.308106 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.308091 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt6zw\" (UniqueName: \"kubernetes.io/projected/11358f6d-0980-4257-b4ff-e0c254bdedde-kube-api-access-dt6zw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7tt5h\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.308177 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.308153 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa42d570-0e77-43a1-960f-7d3ee7ec554b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fa42d570-0e77-43a1-960f-7d3ee7ec554b" (UID: "fa42d570-0e77-43a1-960f-7d3ee7ec554b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:25:36.309974 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.309953 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa42d570-0e77-43a1-960f-7d3ee7ec554b-kube-api-access-xld92" (OuterVolumeSpecName: "kube-api-access-xld92") pod "fa42d570-0e77-43a1-960f-7d3ee7ec554b" (UID: "fa42d570-0e77-43a1-960f-7d3ee7ec554b"). InnerVolumeSpecName "kube-api-access-xld92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:25:36.408494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.408464 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11358f6d-0980-4257-b4ff-e0c254bdedde-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7tt5h\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.408600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.408522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt6zw\" (UniqueName: \"kubernetes.io/projected/11358f6d-0980-4257-b4ff-e0c254bdedde-kube-api-access-dt6zw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7tt5h\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.408600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.408580 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xld92\" (UniqueName: \"kubernetes.io/projected/fa42d570-0e77-43a1-960f-7d3ee7ec554b-kube-api-access-xld92\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:25:36.408672 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.408602 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fa42d570-0e77-43a1-960f-7d3ee7ec554b-extensions-socket-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:25:36.409005 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.408987 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11358f6d-0980-4257-b4ff-e0c254bdedde-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7tt5h\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.417187 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.417167 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt6zw\" (UniqueName: \"kubernetes.io/projected/11358f6d-0980-4257-b4ff-e0c254bdedde-kube-api-access-dt6zw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7tt5h\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.498713 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.498660 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:36.782023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:36.781989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h"] Sep 10 22:25:36.785067 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:25:36.785031 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11358f6d_0980_4257_b4ff_e0c254bdedde.slice/crio-328889d474819133acdefcb36672963695263a3fb4374c98755202ccea649a4b WatchSource:0}: Error finding container 328889d474819133acdefcb36672963695263a3fb4374c98755202ccea649a4b: Status 404 returned error can't find the container with id 328889d474819133acdefcb36672963695263a3fb4374c98755202ccea649a4b Sep 10 22:25:37.139251 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.139163 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" Sep 10 22:25:37.139251 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.139183 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744" event={"ID":"fa42d570-0e77-43a1-960f-7d3ee7ec554b","Type":"ContainerDied","Data":"16c6fcc9a7e2983c3f2858252e69b6db003f948a3e29a3657bf3d6b9924b0fba"} Sep 10 22:25:37.139251 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.139244 2577 scope.go:117] "RemoveContainer" containerID="be434d5730c9ad3de4c0d0fcd36382a34ce478fea0099072af97453cdaf4486b" Sep 10 22:25:37.141255 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.141211 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" event={"ID":"11358f6d-0980-4257-b4ff-e0c254bdedde","Type":"ContainerStarted","Data":"0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6"} Sep 10 22:25:37.141387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.141264 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" event={"ID":"11358f6d-0980-4257-b4ff-e0c254bdedde","Type":"ContainerStarted","Data":"328889d474819133acdefcb36672963695263a3fb4374c98755202ccea649a4b"} Sep 10 22:25:37.141387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.141293 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:25:37.165333 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.165294 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" podStartSLOduration=1.165280481 podStartE2EDuration="1.165280481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:37.163459668 +0000 UTC m=+443.959675145" watchObservedRunningTime="2026-09-10 22:25:37.165280481 +0000 UTC m=+443.961495957" Sep 10 22:25:37.179483 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.179461 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744"] Sep 10 22:25:37.184971 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.184941 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-2j744"] Sep 10 22:25:37.750072 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:37.750040 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa42d570-0e77-43a1-960f-7d3ee7ec554b" path="/var/lib/kubelet/pods/fa42d570-0e77-43a1-960f-7d3ee7ec554b/volumes" Sep 10 22:25:48.150956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:25:48.150923 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:26:08.221445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.221408 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-p5h5k"] Sep 10 22:26:08.225011 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.224995 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.227178 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.227150 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j749g\"" Sep 10 22:26:08.227303 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.227150 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 10 22:26:08.231179 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.231157 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-p5h5k"] Sep 10 22:26:08.282539 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.282507 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-config-file\") pod \"limitador-limitador-8588749977-p5h5k\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.282692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.282636 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs46c\" (UniqueName: \"kubernetes.io/projected/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-kube-api-access-gs46c\") pod \"limitador-limitador-8588749977-p5h5k\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.317662 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.317632 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-p5h5k"] Sep 10 22:26:08.383154 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.383116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-config-file\") pod \"limitador-limitador-8588749977-p5h5k\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.383325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.383221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs46c\" (UniqueName: \"kubernetes.io/projected/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-kube-api-access-gs46c\") pod \"limitador-limitador-8588749977-p5h5k\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.383783 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.383764 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-config-file\") pod \"limitador-limitador-8588749977-p5h5k\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.391194 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.391171 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs46c\" (UniqueName: \"kubernetes.io/projected/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-kube-api-access-gs46c\") pod \"limitador-limitador-8588749977-p5h5k\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.536016 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.535975 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:08.811716 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:08.811681 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-p5h5k"] Sep 10 22:26:08.813791 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:26:08.813766 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6f2541_e733_4b7c_a3c5_4222d4e443e3.slice/crio-e822dbc9253b1108c0abdb9682bab95a8d1b7e224fb81cdfdbdca7289b3f2b0a WatchSource:0}: Error finding container e822dbc9253b1108c0abdb9682bab95a8d1b7e224fb81cdfdbdca7289b3f2b0a: Status 404 returned error can't find the container with id e822dbc9253b1108c0abdb9682bab95a8d1b7e224fb81cdfdbdca7289b3f2b0a Sep 10 22:26:09.325362 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:09.325326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" event={"ID":"9b6f2541-e733-4b7c-a3c5-4222d4e443e3","Type":"ContainerStarted","Data":"e822dbc9253b1108c0abdb9682bab95a8d1b7e224fb81cdfdbdca7289b3f2b0a"} Sep 10 22:26:10.332880 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:10.332842 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" event={"ID":"9b6f2541-e733-4b7c-a3c5-4222d4e443e3","Type":"ContainerStarted","Data":"2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf"} Sep 10 22:26:10.333252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:10.332930 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:10.349133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:10.349086 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" podStartSLOduration=1.274774455 podStartE2EDuration="2.349072176s" totalImagesPullingTime="1.074297721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:26:10.347606992 +0000 UTC m=+477.143822470" watchObservedRunningTime="2026-09-10 22:26:10.349072176 +0000 UTC m=+477.145287652" Sep 10 22:26:21.340206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:21.340170 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:23.062958 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.062920 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-p5h5k"] Sep 10 22:26:23.063362 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.063146 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" podUID="9b6f2541-e733-4b7c-a3c5-4222d4e443e3" containerName="limitador" containerID="cri-o://2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf" gracePeriod=30 Sep 10 22:26:23.611443 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.611419 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:23.724094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.724010 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-config-file\") pod \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " Sep 10 22:26:23.724094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.724050 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs46c\" (UniqueName: \"kubernetes.io/projected/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-kube-api-access-gs46c\") pod \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\" (UID: \"9b6f2541-e733-4b7c-a3c5-4222d4e443e3\") " Sep 10 22:26:23.724397 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.724372 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-config-file" (OuterVolumeSpecName: "config-file") pod "9b6f2541-e733-4b7c-a3c5-4222d4e443e3" (UID: "9b6f2541-e733-4b7c-a3c5-4222d4e443e3"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:26:23.726255 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.726231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-kube-api-access-gs46c" (OuterVolumeSpecName: "kube-api-access-gs46c") pod "9b6f2541-e733-4b7c-a3c5-4222d4e443e3" (UID: "9b6f2541-e733-4b7c-a3c5-4222d4e443e3"). InnerVolumeSpecName "kube-api-access-gs46c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:26:23.825423 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.825397 2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-config-file\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:26:23.825423 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:23.825419 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs46c\" (UniqueName: \"kubernetes.io/projected/9b6f2541-e733-4b7c-a3c5-4222d4e443e3-kube-api-access-gs46c\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:26:24.411045 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.411012 2577 generic.go:356] "Generic (PLEG): container finished" podID="9b6f2541-e733-4b7c-a3c5-4222d4e443e3" containerID="2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf" exitCode=0 Sep 10 22:26:24.411520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.411066 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" Sep 10 22:26:24.411520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.411099 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" event={"ID":"9b6f2541-e733-4b7c-a3c5-4222d4e443e3","Type":"ContainerDied","Data":"2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf"} Sep 10 22:26:24.411520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.411165 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-p5h5k" event={"ID":"9b6f2541-e733-4b7c-a3c5-4222d4e443e3","Type":"ContainerDied","Data":"e822dbc9253b1108c0abdb9682bab95a8d1b7e224fb81cdfdbdca7289b3f2b0a"} Sep 10 22:26:24.411520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.411202 2577 scope.go:117] "RemoveContainer" containerID="2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf" Sep 10 22:26:24.424116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.423015 2577 scope.go:117] "RemoveContainer" containerID="2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf" Sep 10 22:26:24.424481 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:26:24.424231 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf\": container with ID starting with 2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf not found: ID does not exist" containerID="2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf" Sep 10 22:26:24.424481 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.424267 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf"} err="failed to get container status \"2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf\": rpc error: code = NotFound desc = could not find container \"2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf\": container with ID starting with 2b69a932da4cfd435bdb902efa89cb6c953c6ec4eeada2e08cac0efc9f683eaf not found: ID does not exist" Sep 10 22:26:24.426561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.426540 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-p5h5k"] Sep 10 22:26:24.430384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:24.430363 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-p5h5k"] Sep 10 22:26:25.749925 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:25.749893 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6f2541-e733-4b7c-a3c5-4222d4e443e3" path="/var/lib/kubelet/pods/9b6f2541-e733-4b7c-a3c5-4222d4e443e3/volumes" Sep 10 22:26:26.810715 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.810682 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9"] Sep 10 22:26:26.811140 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.811126 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6f2541-e733-4b7c-a3c5-4222d4e443e3" containerName="limitador" Sep 10 22:26:26.811184 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.811142 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6f2541-e733-4b7c-a3c5-4222d4e443e3" containerName="limitador" Sep 10 22:26:26.811239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.811224 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6f2541-e733-4b7c-a3c5-4222d4e443e3" containerName="limitador" Sep 10 22:26:26.815617 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.815599 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:26.817898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.817875 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 22:26:26.818471 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.818452 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 22:26:26.818615 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.818461 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 10 22:26:26.818743 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.818475 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-trhjs\"" Sep 10 22:26:26.823115 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.823093 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9"] Sep 10 22:26:26.950464 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.950436 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d33497e9-eb38-4d74-9d83-576e7cda48b8-data\") pod \"postgres-69fb4594cf-jxfr9\" (UID: \"d33497e9-eb38-4d74-9d83-576e7cda48b8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:26.950587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:26.950489 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmtbd\" (UniqueName: \"kubernetes.io/projected/d33497e9-eb38-4d74-9d83-576e7cda48b8-kube-api-access-pmtbd\") pod \"postgres-69fb4594cf-jxfr9\" (UID: \"d33497e9-eb38-4d74-9d83-576e7cda48b8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:27.051328 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:27.051303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d33497e9-eb38-4d74-9d83-576e7cda48b8-data\") pod \"postgres-69fb4594cf-jxfr9\" (UID: \"d33497e9-eb38-4d74-9d83-576e7cda48b8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:27.051451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:27.051351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmtbd\" (UniqueName: \"kubernetes.io/projected/d33497e9-eb38-4d74-9d83-576e7cda48b8-kube-api-access-pmtbd\") pod \"postgres-69fb4594cf-jxfr9\" (UID: \"d33497e9-eb38-4d74-9d83-576e7cda48b8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:27.051662 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:27.051642 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/d33497e9-eb38-4d74-9d83-576e7cda48b8-data\") pod \"postgres-69fb4594cf-jxfr9\" (UID: \"d33497e9-eb38-4d74-9d83-576e7cda48b8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:27.059704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:27.059670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmtbd\" (UniqueName: \"kubernetes.io/projected/d33497e9-eb38-4d74-9d83-576e7cda48b8-kube-api-access-pmtbd\") pod \"postgres-69fb4594cf-jxfr9\" (UID: \"d33497e9-eb38-4d74-9d83-576e7cda48b8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:27.128145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:27.128083 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:27.400693 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:27.400672 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9"] Sep 10 22:26:27.402591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:26:27.402562 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33497e9_eb38_4d74_9d83_576e7cda48b8.slice/crio-25e0c6b42b8dcc84eeca018bb6cf206ab86b895c3ad35e7e478f0461a0f37a70 WatchSource:0}: Error finding container 25e0c6b42b8dcc84eeca018bb6cf206ab86b895c3ad35e7e478f0461a0f37a70: Status 404 returned error can't find the container with id 25e0c6b42b8dcc84eeca018bb6cf206ab86b895c3ad35e7e478f0461a0f37a70 Sep 10 22:26:27.428400 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:27.428376 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" event={"ID":"d33497e9-eb38-4d74-9d83-576e7cda48b8","Type":"ContainerStarted","Data":"25e0c6b42b8dcc84eeca018bb6cf206ab86b895c3ad35e7e478f0461a0f37a70"} Sep 10 22:26:32.464369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:32.464329 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" event={"ID":"d33497e9-eb38-4d74-9d83-576e7cda48b8","Type":"ContainerStarted","Data":"965b74e1c25c4119b6505b76c54b65573e5d15172ff499f0c1d53be544a6071e"} Sep 10 22:26:32.464725 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:32.464446 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:32.482065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:32.482022 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" podStartSLOduration=1.528495989 podStartE2EDuration="6.482004295s" totalImagesPullingTime="4.953508306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:26:32.479881192 +0000 UTC m=+499.276096669" watchObservedRunningTime="2026-09-10 22:26:32.482004295 +0000 UTC m=+499.278219774" Sep 10 22:26:38.499081 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:38.499045 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jxfr9" Sep 10 22:26:46.009151 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.009116 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-656d98b4cc-6762z"] Sep 10 22:26:46.027275 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.027245 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656d98b4cc-6762z"] Sep 10 22:26:46.027459 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.027344 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.029774 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.029746 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cc2r4\"" Sep 10 22:26:46.029959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.029786 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 10 22:26:46.030064 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.030044 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 10 22:26:46.031150 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.031130 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 10 22:26:46.126778 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.126754 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-webhook-cert\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.126950 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.126790 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vrzp\" (UniqueName: \"kubernetes.io/projected/924d3111-74fa-40cd-89e0-ad0f28e7df4d-kube-api-access-2vrzp\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.127003 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.126943 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-metrics-certs\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.227542 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.227512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-metrics-certs\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.227841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.227566 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-webhook-cert\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.227841 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.227637 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vrzp\" (UniqueName: \"kubernetes.io/projected/924d3111-74fa-40cd-89e0-ad0f28e7df4d-kube-api-access-2vrzp\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.230324 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.230302 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-metrics-certs\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.230465 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.230393 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-webhook-cert\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.236349 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.236326 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vrzp\" (UniqueName: \"kubernetes.io/projected/924d3111-74fa-40cd-89e0-ad0f28e7df4d-kube-api-access-2vrzp\") pod \"maas-controller-656d98b4cc-6762z\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.339085 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.339057 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:46.614490 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:46.614463 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656d98b4cc-6762z"] Sep 10 22:26:46.616560 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:26:46.616530 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924d3111_74fa_40cd_89e0_ad0f28e7df4d.slice/crio-cbd8409e3f77ea0b6a00066210d71fd47e5d0667a6110ae1890b707c9aec00f9 WatchSource:0}: Error finding container cbd8409e3f77ea0b6a00066210d71fd47e5d0667a6110ae1890b707c9aec00f9: Status 404 returned error can't find the container with id cbd8409e3f77ea0b6a00066210d71fd47e5d0667a6110ae1890b707c9aec00f9 Sep 10 22:26:47.547718 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:47.547661 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656d98b4cc-6762z" event={"ID":"924d3111-74fa-40cd-89e0-ad0f28e7df4d","Type":"ContainerStarted","Data":"cbd8409e3f77ea0b6a00066210d71fd47e5d0667a6110ae1890b707c9aec00f9"} Sep 10 22:26:49.562716 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:49.562681 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656d98b4cc-6762z" event={"ID":"924d3111-74fa-40cd-89e0-ad0f28e7df4d","Type":"ContainerStarted","Data":"a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc"} Sep 10 22:26:49.563115 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:49.562810 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:26:49.580015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:26:49.579971 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-656d98b4cc-6762z" podStartSLOduration=2.252947074 podStartE2EDuration="4.579956995s" totalImagesPullingTime="2.327009921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:26:49.5785501 +0000 UTC m=+516.374765577" watchObservedRunningTime="2026-09-10 22:26:49.579956995 +0000 UTC m=+516.376172471" Sep 10 22:27:00.273460 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.273425 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-656d98b4cc-6762z"] Sep 10 22:27:00.273930 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.273656 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-656d98b4cc-6762z" podUID="924d3111-74fa-40cd-89e0-ad0f28e7df4d" containerName="manager" containerID="cri-o://a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc" gracePeriod=30 Sep 10 22:27:00.275259 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.275234 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:27:00.526305 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.526244 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:27:00.549798 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.549772 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vrzp\" (UniqueName: \"kubernetes.io/projected/924d3111-74fa-40cd-89e0-ad0f28e7df4d-kube-api-access-2vrzp\") pod \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " Sep 10 22:27:00.549798 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.549802 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-metrics-certs\") pod \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " Sep 10 22:27:00.550049 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.549956 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-webhook-cert\") pod \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\" (UID: \"924d3111-74fa-40cd-89e0-ad0f28e7df4d\") " Sep 10 22:27:00.552104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.552067 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "924d3111-74fa-40cd-89e0-ad0f28e7df4d" (UID: "924d3111-74fa-40cd-89e0-ad0f28e7df4d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:27:00.552237 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.552142 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "924d3111-74fa-40cd-89e0-ad0f28e7df4d" (UID: "924d3111-74fa-40cd-89e0-ad0f28e7df4d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:27:00.552237 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.552142 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/924d3111-74fa-40cd-89e0-ad0f28e7df4d-kube-api-access-2vrzp" (OuterVolumeSpecName: "kube-api-access-2vrzp") pod "924d3111-74fa-40cd-89e0-ad0f28e7df4d" (UID: "924d3111-74fa-40cd-89e0-ad0f28e7df4d"). InnerVolumeSpecName "kube-api-access-2vrzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:27:00.625082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.625049 2577 generic.go:356] "Generic (PLEG): container finished" podID="924d3111-74fa-40cd-89e0-ad0f28e7df4d" containerID="a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc" exitCode=0 Sep 10 22:27:00.625216 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.625093 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656d98b4cc-6762z" Sep 10 22:27:00.625216 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.625134 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656d98b4cc-6762z" event={"ID":"924d3111-74fa-40cd-89e0-ad0f28e7df4d","Type":"ContainerDied","Data":"a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc"} Sep 10 22:27:00.625216 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.625188 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656d98b4cc-6762z" event={"ID":"924d3111-74fa-40cd-89e0-ad0f28e7df4d","Type":"ContainerDied","Data":"cbd8409e3f77ea0b6a00066210d71fd47e5d0667a6110ae1890b707c9aec00f9"} Sep 10 22:27:00.625334 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.625227 2577 scope.go:117] "RemoveContainer" containerID="a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc" Sep 10 22:27:00.635338 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.635322 2577 scope.go:117] "RemoveContainer" containerID="a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc" Sep 10 22:27:00.635606 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:27:00.635583 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc\": container with ID starting with a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc not found: ID does not exist" containerID="a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc" Sep 10 22:27:00.635712 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.635613 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc"} err="failed to get container status \"a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc\": rpc error: code = NotFound desc = could not find container \"a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc\": container with ID starting with a1877f444da84d4bff3baebcee97bbfecf418315eaee7c8af1ae211308670edc not found: ID does not exist" Sep 10 22:27:00.646962 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.646941 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-656d98b4cc-6762z"] Sep 10 22:27:00.651134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.651115 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vrzp\" (UniqueName: \"kubernetes.io/projected/924d3111-74fa-40cd-89e0-ad0f28e7df4d-kube-api-access-2vrzp\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:27:00.651218 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.651137 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:27:00.651218 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.651147 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/924d3111-74fa-40cd-89e0-ad0f28e7df4d-webhook-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:27:00.652428 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:00.652409 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-656d98b4cc-6762z"] Sep 10 22:27:01.749485 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:01.749452 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="924d3111-74fa-40cd-89e0-ad0f28e7df4d" path="/var/lib/kubelet/pods/924d3111-74fa-40cd-89e0-ad0f28e7df4d/volumes" Sep 10 22:27:22.978974 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.978933 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s"] Sep 10 22:27:22.979657 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.979595 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="924d3111-74fa-40cd-89e0-ad0f28e7df4d" containerName="manager" Sep 10 22:27:22.979657 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.979618 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="924d3111-74fa-40cd-89e0-ad0f28e7df4d" containerName="manager" Sep 10 22:27:22.979909 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.979721 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="924d3111-74fa-40cd-89e0-ad0f28e7df4d" containerName="manager" Sep 10 22:27:22.984991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.984968 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:22.987225 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.987204 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 22:27:22.987324 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.987277 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9s5hd\"" Sep 10 22:27:22.993165 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:22.993145 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s"] Sep 10 22:27:23.012790 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.012755 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-t7dns"] Sep 10 22:27:23.015987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.015970 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.030333 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.030312 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-t7dns"] Sep 10 22:27:23.143026 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.142994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ee3d49-6550-42b3-bafc-c70e46d7f35b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jcw7s\" (UID: \"43ee3d49-6550-42b3-bafc-c70e46d7f35b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:23.143185 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.143035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhcbh\" (UniqueName: \"kubernetes.io/projected/43ee3d49-6550-42b3-bafc-c70e46d7f35b-kube-api-access-rhcbh\") pod \"payload-pre-processing-59fbc47bf8-jcw7s\" (UID: \"43ee3d49-6550-42b3-bafc-c70e46d7f35b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:23.143185 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.143074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/471c8294-5aad-47d3-8392-d937af251324-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-t7dns\" (UID: \"471c8294-5aad-47d3-8392-d937af251324\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.143185 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.143094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnmrm\" (UniqueName: \"kubernetes.io/projected/471c8294-5aad-47d3-8392-d937af251324-kube-api-access-pnmrm\") pod \"payload-processing-5b54b7fbb7-t7dns\" (UID: \"471c8294-5aad-47d3-8392-d937af251324\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.244414 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.244340 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhcbh\" (UniqueName: \"kubernetes.io/projected/43ee3d49-6550-42b3-bafc-c70e46d7f35b-kube-api-access-rhcbh\") pod \"payload-pre-processing-59fbc47bf8-jcw7s\" (UID: \"43ee3d49-6550-42b3-bafc-c70e46d7f35b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:23.244414 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.244398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/471c8294-5aad-47d3-8392-d937af251324-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-t7dns\" (UID: \"471c8294-5aad-47d3-8392-d937af251324\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.244623 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.244579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnmrm\" (UniqueName: \"kubernetes.io/projected/471c8294-5aad-47d3-8392-d937af251324-kube-api-access-pnmrm\") pod \"payload-processing-5b54b7fbb7-t7dns\" (UID: \"471c8294-5aad-47d3-8392-d937af251324\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.244725 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.244706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ee3d49-6550-42b3-bafc-c70e46d7f35b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jcw7s\" (UID: \"43ee3d49-6550-42b3-bafc-c70e46d7f35b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:23.245071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.245051 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/471c8294-5aad-47d3-8392-d937af251324-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-t7dns\" (UID: \"471c8294-5aad-47d3-8392-d937af251324\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.245344 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.245326 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43ee3d49-6550-42b3-bafc-c70e46d7f35b-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-jcw7s\" (UID: \"43ee3d49-6550-42b3-bafc-c70e46d7f35b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:23.253472 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.253451 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnmrm\" (UniqueName: \"kubernetes.io/projected/471c8294-5aad-47d3-8392-d937af251324-kube-api-access-pnmrm\") pod \"payload-processing-5b54b7fbb7-t7dns\" (UID: \"471c8294-5aad-47d3-8392-d937af251324\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.253645 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.253626 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhcbh\" (UniqueName: \"kubernetes.io/projected/43ee3d49-6550-42b3-bafc-c70e46d7f35b-kube-api-access-rhcbh\") pod \"payload-pre-processing-59fbc47bf8-jcw7s\" (UID: \"43ee3d49-6550-42b3-bafc-c70e46d7f35b\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:23.298167 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.298140 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:23.325699 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.325673 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:23.574370 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.574346 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s"] Sep 10 22:27:23.575752 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:27:23.575724 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43ee3d49_6550_42b3_bafc_c70e46d7f35b.slice/crio-746375c4b67101b5a15bef4a9ff46715c045857a211adbe5ea128419cf50e7a3 WatchSource:0}: Error finding container 746375c4b67101b5a15bef4a9ff46715c045857a211adbe5ea128419cf50e7a3: Status 404 returned error can't find the container with id 746375c4b67101b5a15bef4a9ff46715c045857a211adbe5ea128419cf50e7a3 Sep 10 22:27:23.604993 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.604967 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-t7dns"] Sep 10 22:27:23.606484 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:27:23.606458 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471c8294_5aad_47d3_8392_d937af251324.slice/crio-90e387de34081b05710d568a98f7155952084ee1492d18ffb8479ccca29bc988 WatchSource:0}: Error finding container 90e387de34081b05710d568a98f7155952084ee1492d18ffb8479ccca29bc988: Status 404 returned error can't find the container with id 90e387de34081b05710d568a98f7155952084ee1492d18ffb8479ccca29bc988 Sep 10 22:27:23.759505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.759466 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" event={"ID":"471c8294-5aad-47d3-8392-d937af251324","Type":"ContainerStarted","Data":"90e387de34081b05710d568a98f7155952084ee1492d18ffb8479ccca29bc988"} Sep 10 22:27:23.760704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:23.760671 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" event={"ID":"43ee3d49-6550-42b3-bafc-c70e46d7f35b","Type":"ContainerStarted","Data":"746375c4b67101b5a15bef4a9ff46715c045857a211adbe5ea128419cf50e7a3"} Sep 10 22:27:25.776748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:25.776714 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" event={"ID":"471c8294-5aad-47d3-8392-d937af251324","Type":"ContainerStarted","Data":"8e21fced910bf388a9aff754b4f441374f70c96102e69124be88b50f15aea55d"} Sep 10 22:27:25.777193 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:25.776842 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:25.778576 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:25.778551 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" event={"ID":"43ee3d49-6550-42b3-bafc-c70e46d7f35b","Type":"ContainerStarted","Data":"58c4772f8231d33396fda1af46383992a5b63d4b6084b02adf3671ddb785c78e"} Sep 10 22:27:25.778699 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:25.778640 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:25.791892 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:25.791801 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" podStartSLOduration=2.044393941 podStartE2EDuration="3.791785392s" totalImagesPullingTime="1.747391451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:27:25.791677017 +0000 UTC m=+552.587892491" watchObservedRunningTime="2026-09-10 22:27:25.791785392 +0000 UTC m=+552.588000874" Sep 10 22:27:25.805711 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:25.805664 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" podStartSLOduration=2.020805302 podStartE2EDuration="3.805648458s" totalImagesPullingTime="1.784843156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:27:25.805408615 +0000 UTC m=+552.601624093" watchObservedRunningTime="2026-09-10 22:27:25.805648458 +0000 UTC m=+552.601863936" Sep 10 22:27:36.786757 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:36.786723 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-jcw7s" Sep 10 22:27:36.787203 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:36.786792 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-t7dns" Sep 10 22:27:49.536470 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.536393 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l"] Sep 10 22:27:49.540646 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.540622 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.543417 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.543394 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m62lm\"" Sep 10 22:27:49.543538 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.543416 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 22:27:49.543538 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.543393 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 10 22:27:49.543538 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.543402 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 22:27:49.549635 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.549614 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l"] Sep 10 22:27:49.676386 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.676353 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.676541 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.676417 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.676541 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.676490 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.676541 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.676525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.676651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.676546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.676651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.676571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hzzh\" (UniqueName: \"kubernetes.io/projected/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-kube-api-access-9hzzh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.777886 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.777853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.777913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.777938 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.777967 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.777984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.778004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hzzh\" (UniqueName: \"kubernetes.io/projected/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-kube-api-access-9hzzh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.778247 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778326 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.778301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.778378 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.778341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.780528 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.780499 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.780757 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.780740 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.786731 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.786672 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hzzh\" (UniqueName: \"kubernetes.io/projected/ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7-kube-api-access-9hzzh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-87t4l\" (UID: \"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:49.851930 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:49.851893 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:50.149135 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:50.149112 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l"] Sep 10 22:27:50.151169 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:27:50.151140 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec33c8f0_a7f8_4fe9_9459_c0b85cf5ddc7.slice/crio-7408764bcc54da4d6fbf5d3f9108edd314b163c4a7ef7d92cfccd1d10fbe5f55 WatchSource:0}: Error finding container 7408764bcc54da4d6fbf5d3f9108edd314b163c4a7ef7d92cfccd1d10fbe5f55: Status 404 returned error can't find the container with id 7408764bcc54da4d6fbf5d3f9108edd314b163c4a7ef7d92cfccd1d10fbe5f55 Sep 10 22:27:50.922641 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:50.922586 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" event={"ID":"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7","Type":"ContainerStarted","Data":"7408764bcc54da4d6fbf5d3f9108edd314b163c4a7ef7d92cfccd1d10fbe5f55"} Sep 10 22:27:51.929792 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:51.929754 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" event={"ID":"ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7","Type":"ContainerStarted","Data":"861438985f7c5250888df82ec94d4be528aa5da8d88edd1135f00a47a72a18cb"} Sep 10 22:27:51.948084 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:51.948037 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" podStartSLOduration=1.979109751 podStartE2EDuration="2.948021608s" totalImagesPullingTime="968.911857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:27:51.945691843 +0000 UTC m=+578.741907348" watchObservedRunningTime="2026-09-10 22:27:51.948021608 +0000 UTC m=+578.744237085" Sep 10 22:27:59.852295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:59.852258 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:59.852731 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:59.852319 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:59.865123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:59.865100 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:27:59.987261 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:27:59.987234 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-87t4l" Sep 10 22:28:13.663947 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:28:13.663922 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:28:13.664404 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:28:13.663986 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:30:00.148199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.148166 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk"] Sep 10 22:30:00.151650 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.151634 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" Sep 10 22:30:00.153975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.153954 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bltkc\"" Sep 10 22:30:00.160551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.160527 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk"] Sep 10 22:30:00.163991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.163971 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" Sep 10 22:30:00.446300 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.446273 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk"] Sep 10 22:30:00.448481 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:00.448454 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c445bf_9821_4312_9a2f_e800edfa3263.slice/crio-081984f906118cfc59f7777dd91f9a75c476caf30b45f85c27bdb9de75fce624 WatchSource:0}: Error finding container 081984f906118cfc59f7777dd91f9a75c476caf30b45f85c27bdb9de75fce624: Status 404 returned error can't find the container with id 081984f906118cfc59f7777dd91f9a75c476caf30b45f85c27bdb9de75fce624 Sep 10 22:30:00.450212 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.450193 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:30:00.670625 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:00.670590 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" event={"ID":"12c445bf-9821-4312-9a2f-e800edfa3263","Type":"ContainerStarted","Data":"081984f906118cfc59f7777dd91f9a75c476caf30b45f85c27bdb9de75fce624"} Sep 10 22:30:03.696571 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:03.696532 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" event={"ID":"12c445bf-9821-4312-9a2f-e800edfa3263","Type":"ContainerStarted","Data":"c0b954b5e8cad307a1c744342342dedc908f89e7e8b4551c2fd9892a714d4523"} Sep 10 22:30:03.713600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:03.713543 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" podStartSLOduration=1.5020682669999998 podStartE2EDuration="3.713527731s" totalImagesPullingTime="2.211459464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:03.711337252 +0000 UTC m=+710.507552728" watchObservedRunningTime="2026-09-10 22:30:03.713527731 +0000 UTC m=+710.509743213" Sep 10 22:30:04.703950 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:04.703875 2577 generic.go:356] "Generic (PLEG): container finished" podID="12c445bf-9821-4312-9a2f-e800edfa3263" containerID="c0b954b5e8cad307a1c744342342dedc908f89e7e8b4551c2fd9892a714d4523" exitCode=0 Sep 10 22:30:04.704412 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:04.703962 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" event={"ID":"12c445bf-9821-4312-9a2f-e800edfa3263","Type":"ContainerDied","Data":"c0b954b5e8cad307a1c744342342dedc908f89e7e8b4551c2fd9892a714d4523"} Sep 10 22:30:05.849445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:05.849411 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" Sep 10 22:30:06.717327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:06.717293 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" Sep 10 22:30:06.717517 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:06.717350 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817990-wbkzk" event={"ID":"12c445bf-9821-4312-9a2f-e800edfa3263","Type":"ContainerDied","Data":"081984f906118cfc59f7777dd91f9a75c476caf30b45f85c27bdb9de75fce624"} Sep 10 22:30:06.717517 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:06.717444 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="081984f906118cfc59f7777dd91f9a75c476caf30b45f85c27bdb9de75fce624" Sep 10 22:30:07.277017 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.276979 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9"] Sep 10 22:30:07.277406 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.277394 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c445bf-9821-4312-9a2f-e800edfa3263" containerName="cleanup" Sep 10 22:30:07.277451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.277408 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c445bf-9821-4312-9a2f-e800edfa3263" containerName="cleanup" Sep 10 22:30:07.277490 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.277483 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c445bf-9821-4312-9a2f-e800edfa3263" containerName="cleanup" Sep 10 22:30:07.280449 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.280425 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.283024 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.283003 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ddf324-serving-cert\"" Sep 10 22:30:07.283183 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.283011 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 22:30:07.283254 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.283010 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ljsfd\"" Sep 10 22:30:07.296725 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.296689 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9"] Sep 10 22:30:07.376006 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.375966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-metrics-certs\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.376190 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.376018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eddebc97-2248-457a-81d4-f7048b1372c6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.376190 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.376155 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lggf\" (UniqueName: \"kubernetes.io/projected/eddebc97-2248-457a-81d4-f7048b1372c6-kube-api-access-5lggf\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.376290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.376196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-maas-api-tls\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.476827 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.476794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-metrics-certs\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.476991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.476861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eddebc97-2248-457a-81d4-f7048b1372c6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.476991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.476929 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lggf\" (UniqueName: \"kubernetes.io/projected/eddebc97-2248-457a-81d4-f7048b1372c6-kube-api-access-5lggf\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.476991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.476957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-maas-api-tls\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.477690 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.477600 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eddebc97-2248-457a-81d4-f7048b1372c6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.479375 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.479351 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-metrics-certs\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.479506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.479379 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-maas-api-tls\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.489753 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.489726 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lggf\" (UniqueName: \"kubernetes.io/projected/eddebc97-2248-457a-81d4-f7048b1372c6-kube-api-access-5lggf\") pod \"maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.593711 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.593643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:07.874398 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.874364 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9"] Sep 10 22:30:07.876154 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:07.876122 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeddebc97_2248_457a_81d4_f7048b1372c6.slice/crio-7152b89073b43f01affa1ad56a45a50673cc498838925b4b1ec80eb66fc3a04d WatchSource:0}: Error finding container 7152b89073b43f01affa1ad56a45a50673cc498838925b4b1ec80eb66fc3a04d: Status 404 returned error can't find the container with id 7152b89073b43f01affa1ad56a45a50673cc498838925b4b1ec80eb66fc3a04d Sep 10 22:30:07.962714 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.962685 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4"] Sep 10 22:30:07.967128 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.967111 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:07.969216 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.969188 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-ddf324-dockercfg-mbrlq\"" Sep 10 22:30:07.969216 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.969209 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-ddf324\"" Sep 10 22:30:07.978931 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:07.978907 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4"] Sep 10 22:30:08.082532 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.082504 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jftdm\" (UniqueName: \"kubernetes.io/projected/0b1331ab-b40b-47c2-8098-0f0cdd09858d-kube-api-access-jftdm\") pod \"payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.082696 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.082537 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b1331ab-b40b-47c2-8098-0f0cdd09858d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.183022 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.182941 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jftdm\" (UniqueName: \"kubernetes.io/projected/0b1331ab-b40b-47c2-8098-0f0cdd09858d-kube-api-access-jftdm\") pod \"payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.183022 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.182979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b1331ab-b40b-47c2-8098-0f0cdd09858d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.183684 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.183648 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b1331ab-b40b-47c2-8098-0f0cdd09858d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.191753 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.191729 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jftdm\" (UniqueName: \"kubernetes.io/projected/0b1331ab-b40b-47c2-8098-0f0cdd09858d-kube-api-access-jftdm\") pod \"payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.278712 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.278686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.555454 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.555428 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4"] Sep 10 22:30:08.557055 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:08.557028 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1331ab_b40b_47c2_8098_0f0cdd09858d.slice/crio-20eb0f76b9590b42f98082b9e5dd151ef57870a92bec74df416ace0228fcfb73 WatchSource:0}: Error finding container 20eb0f76b9590b42f98082b9e5dd151ef57870a92bec74df416ace0228fcfb73: Status 404 returned error can't find the container with id 20eb0f76b9590b42f98082b9e5dd151ef57870a92bec74df416ace0228fcfb73 Sep 10 22:30:08.735308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.735270 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" event={"ID":"eddebc97-2248-457a-81d4-f7048b1372c6","Type":"ContainerStarted","Data":"7152b89073b43f01affa1ad56a45a50673cc498838925b4b1ec80eb66fc3a04d"} Sep 10 22:30:08.737286 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.737251 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" event={"ID":"0b1331ab-b40b-47c2-8098-0f0cdd09858d","Type":"ContainerStarted","Data":"3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29"} Sep 10 22:30:08.737402 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.737303 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" event={"ID":"0b1331ab-b40b-47c2-8098-0f0cdd09858d","Type":"ContainerStarted","Data":"20eb0f76b9590b42f98082b9e5dd151ef57870a92bec74df416ace0228fcfb73"} Sep 10 22:30:08.737402 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.737346 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:08.754263 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:08.754212 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" podStartSLOduration=1.754197016 podStartE2EDuration="1.754197016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:08.753011582 +0000 UTC m=+715.549227060" watchObservedRunningTime="2026-09-10 22:30:08.754197016 +0000 UTC m=+715.550412487" Sep 10 22:30:09.749588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:09.749512 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:09.749588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:09.749559 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" event={"ID":"eddebc97-2248-457a-81d4-f7048b1372c6","Type":"ContainerStarted","Data":"d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac"} Sep 10 22:30:09.770381 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:09.770329 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" podStartSLOduration=1.2158030420000001 podStartE2EDuration="2.770309258s" totalImagesPullingTime="1.554506216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:09.766583232 +0000 UTC m=+716.562798711" watchObservedRunningTime="2026-09-10 22:30:09.770309258 +0000 UTC m=+716.566524740" Sep 10 22:30:15.513979 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.513944 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6"] Sep 10 22:30:15.518143 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.518124 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.520900 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.520878 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9005ccc0\"" Sep 10 22:30:15.521365 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.521348 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9005ccc0-dockercfg-55jlz\"" Sep 10 22:30:15.528164 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.528138 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6"] Sep 10 22:30:15.575563 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.575530 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k"] Sep 10 22:30:15.579310 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.579288 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:15.589781 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.589758 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k"] Sep 10 22:30:15.655417 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.655380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-plugins-config-volume\") pod \"payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.655595 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.655456 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbwxw\" (UniqueName: \"kubernetes.io/projected/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-kube-api-access-dbwxw\") pod \"payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.756978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.756940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d366cde3-64af-4d2d-b2d4-f0fc3d984678-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:15.757168 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.757097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcp56\" (UniqueName: \"kubernetes.io/projected/d366cde3-64af-4d2d-b2d4-f0fc3d984678-kube-api-access-qcp56\") pod \"payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:15.757168 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.757156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-plugins-config-volume\") pod \"payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.757296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.757273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbwxw\" (UniqueName: \"kubernetes.io/projected/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-kube-api-access-dbwxw\") pod \"payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.757960 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.757933 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-plugins-config-volume\") pod \"payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.758409 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.758385 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:30:15.765140 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.765084 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbwxw\" (UniqueName: \"kubernetes.io/projected/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-kube-api-access-dbwxw\") pod \"payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.830262 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.830229 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:15.858829 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.858766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d366cde3-64af-4d2d-b2d4-f0fc3d984678-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:15.858992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.858837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcp56\" (UniqueName: \"kubernetes.io/projected/d366cde3-64af-4d2d-b2d4-f0fc3d984678-kube-api-access-qcp56\") pod \"payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:15.859552 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.859532 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d366cde3-64af-4d2d-b2d4-f0fc3d984678-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:15.866991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.866966 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcp56\" (UniqueName: \"kubernetes.io/projected/d366cde3-64af-4d2d-b2d4-f0fc3d984678-kube-api-access-qcp56\") pod \"payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:15.892860 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:15.892570 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:16.128459 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.128433 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6"] Sep 10 22:30:16.129544 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:16.129513 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06b2ce7_b656_4cb3_b8a0_0ea096acfc1d.slice/crio-3018052ddd3cfd7b6c75f194a0eda5e6481e8143a41ba00d8e0ff3c58ddf928f WatchSource:0}: Error finding container 3018052ddd3cfd7b6c75f194a0eda5e6481e8143a41ba00d8e0ff3c58ddf928f: Status 404 returned error can't find the container with id 3018052ddd3cfd7b6c75f194a0eda5e6481e8143a41ba00d8e0ff3c58ddf928f Sep 10 22:30:16.188014 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.187979 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k"] Sep 10 22:30:16.192681 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:16.192645 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd366cde3_64af_4d2d_b2d4_f0fc3d984678.slice/crio-f3cf1f48c0a8c253716de2cb5460162dbd85766352c0022c99b32156f72d2ff4 WatchSource:0}: Error finding container f3cf1f48c0a8c253716de2cb5460162dbd85766352c0022c99b32156f72d2ff4: Status 404 returned error can't find the container with id f3cf1f48c0a8c253716de2cb5460162dbd85766352c0022c99b32156f72d2ff4 Sep 10 22:30:16.790505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.790233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" event={"ID":"d366cde3-64af-4d2d-b2d4-f0fc3d984678","Type":"ContainerStarted","Data":"f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380"} Sep 10 22:30:16.790505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.790295 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" event={"ID":"d366cde3-64af-4d2d-b2d4-f0fc3d984678","Type":"ContainerStarted","Data":"f3cf1f48c0a8c253716de2cb5460162dbd85766352c0022c99b32156f72d2ff4"} Sep 10 22:30:16.790505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.790328 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:16.792136 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.792108 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" event={"ID":"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d","Type":"ContainerStarted","Data":"a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1"} Sep 10 22:30:16.792274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.792151 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" event={"ID":"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d","Type":"ContainerStarted","Data":"3018052ddd3cfd7b6c75f194a0eda5e6481e8143a41ba00d8e0ff3c58ddf928f"} Sep 10 22:30:16.792274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.792185 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:16.831256 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.831201 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" podStartSLOduration=1.831185988 podStartE2EDuration="1.831185988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:16.829524891 +0000 UTC m=+723.625740368" watchObservedRunningTime="2026-09-10 22:30:16.831185988 +0000 UTC m=+723.627401486" Sep 10 22:30:16.869387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.869345 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" podStartSLOduration=1.8693293720000002 podStartE2EDuration="1.869329372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:16.863941178 +0000 UTC m=+723.660156655" watchObservedRunningTime="2026-09-10 22:30:16.869329372 +0000 UTC m=+723.665544853" Sep 10 22:30:16.893898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.893871 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd"] Sep 10 22:30:16.897481 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.897462 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:16.900655 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.900632 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-cc3245-dockercfg-whjlm\"" Sep 10 22:30:16.900864 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.900847 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-cc3245\"" Sep 10 22:30:16.919978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:16.919956 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd"] Sep 10 22:30:17.068760 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.068684 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjnjt\" (UniqueName: \"kubernetes.io/projected/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-kube-api-access-cjnjt\") pod \"payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.068760 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.068726 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.169878 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.169848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjnjt\" (UniqueName: \"kubernetes.io/projected/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-kube-api-access-cjnjt\") pod \"payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.170061 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.169900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.170614 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.170587 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.181757 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.181737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjnjt\" (UniqueName: \"kubernetes.io/projected/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-kube-api-access-cjnjt\") pod \"payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.209233 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.209211 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.495396 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.495366 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd"] Sep 10 22:30:17.497011 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:17.496979 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dfdb48a_069d_4b68_88e6_74ba6a3eecbf.slice/crio-46df46e6af5a19608bab65e816ef6a66f2404426f131267831728dd9853517ed WatchSource:0}: Error finding container 46df46e6af5a19608bab65e816ef6a66f2404426f131267831728dd9853517ed: Status 404 returned error can't find the container with id 46df46e6af5a19608bab65e816ef6a66f2404426f131267831728dd9853517ed Sep 10 22:30:17.800423 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.800386 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" event={"ID":"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf","Type":"ContainerStarted","Data":"30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129"} Sep 10 22:30:17.800885 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.800447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" event={"ID":"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf","Type":"ContainerStarted","Data":"46df46e6af5a19608bab65e816ef6a66f2404426f131267831728dd9853517ed"} Sep 10 22:30:17.800979 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.800912 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:17.823073 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:17.823028 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" podStartSLOduration=1.8230123059999999 podStartE2EDuration="1.823012306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:17.820325961 +0000 UTC m=+724.616541437" watchObservedRunningTime="2026-09-10 22:30:17.823012306 +0000 UTC m=+724.619227783" Sep 10 22:30:19.751004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:19.750972 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:30:27.803027 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:27.802993 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:27.803610 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:27.803078 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:29.819689 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:29.819646 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:30:36.710222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.710186 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j"] Sep 10 22:30:36.718337 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.718314 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.720316 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.720292 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d1a134-serving-cert\"" Sep 10 22:30:36.726589 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.726564 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j"] Sep 10 22:30:36.755231 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.755203 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.755366 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.755238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.755366 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.755304 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckdkg\" (UniqueName: \"kubernetes.io/projected/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-kube-api-access-ckdkg\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.755366 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.755338 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-metrics-certs\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.856040 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.856005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.856040 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.856041 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.856292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.856086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckdkg\" (UniqueName: \"kubernetes.io/projected/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-kube-api-access-ckdkg\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.856292 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:30:36.856181 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-serving-cert: secret "maas-api-e2e-inf-b-d1a134-serving-cert" not found Sep 10 22:30:36.856292 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:30:36.856249 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls podName:4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12 nodeName:}" failed. No retries permitted until 2026-09-10 22:30:37.356229223 +0000 UTC m=+744.152444677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls") pod "maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" (UID: "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12") : secret "maas-api-e2e-inf-b-d1a134-serving-cert" not found Sep 10 22:30:36.856292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.856271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-metrics-certs\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.856618 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.856596 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.858981 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.858950 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-metrics-certs\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:36.866411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:36.866384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckdkg\" (UniqueName: \"kubernetes.io/projected/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-kube-api-access-ckdkg\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:37.349271 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.349233 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4"] Sep 10 22:30:37.353104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.353083 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.356123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.356095 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d1a134-dockercfg-44nv2\"" Sep 10 22:30:37.356343 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.356329 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d1a134\"" Sep 10 22:30:37.361371 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.361351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:37.364400 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.364377 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls\") pod \"maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:37.365989 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.365969 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4"] Sep 10 22:30:37.462890 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.462856 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01390529-63ad-4e84-a073-e29cc04b3192-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.463048 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.462914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jccjd\" (UniqueName: \"kubernetes.io/projected/01390529-63ad-4e84-a073-e29cc04b3192-kube-api-access-jccjd\") pod \"payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.564735 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.564696 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01390529-63ad-4e84-a073-e29cc04b3192-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.565000 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.564773 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jccjd\" (UniqueName: \"kubernetes.io/projected/01390529-63ad-4e84-a073-e29cc04b3192-kube-api-access-jccjd\") pod \"payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.566144 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.566115 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01390529-63ad-4e84-a073-e29cc04b3192-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.573180 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.573157 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jccjd\" (UniqueName: \"kubernetes.io/projected/01390529-63ad-4e84-a073-e29cc04b3192-kube-api-access-jccjd\") pod \"payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.632784 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.632700 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:37.664992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.664962 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:37.910643 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.910616 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j"] Sep 10 22:30:37.911516 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:37.911491 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e90ce12_c4b1_4ab9_83a7_2c1e7e6cca12.slice/crio-55bfc1ac9f91265170016c8b93f978480355874b03d3b72f8a32f1a2d0d246d9 WatchSource:0}: Error finding container 55bfc1ac9f91265170016c8b93f978480355874b03d3b72f8a32f1a2d0d246d9: Status 404 returned error can't find the container with id 55bfc1ac9f91265170016c8b93f978480355874b03d3b72f8a32f1a2d0d246d9 Sep 10 22:30:37.929539 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.929505 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" event={"ID":"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12","Type":"ContainerStarted","Data":"55bfc1ac9f91265170016c8b93f978480355874b03d3b72f8a32f1a2d0d246d9"} Sep 10 22:30:37.952874 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:37.952853 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4"] Sep 10 22:30:37.954333 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:37.954312 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01390529_63ad_4e84_a073_e29cc04b3192.slice/crio-b78b75dd7bf8aab1fb793fc84571708741e14285143eb9810daaed0dfe8ce570 WatchSource:0}: Error finding container b78b75dd7bf8aab1fb793fc84571708741e14285143eb9810daaed0dfe8ce570: Status 404 returned error can't find the container with id b78b75dd7bf8aab1fb793fc84571708741e14285143eb9810daaed0dfe8ce570 Sep 10 22:30:38.937588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:38.937543 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" event={"ID":"01390529-63ad-4e84-a073-e29cc04b3192","Type":"ContainerStarted","Data":"abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6"} Sep 10 22:30:38.938060 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:38.937615 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" event={"ID":"01390529-63ad-4e84-a073-e29cc04b3192","Type":"ContainerStarted","Data":"b78b75dd7bf8aab1fb793fc84571708741e14285143eb9810daaed0dfe8ce570"} Sep 10 22:30:38.938060 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:38.937669 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:38.939335 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:38.939312 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" event={"ID":"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12","Type":"ContainerStarted","Data":"f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b"} Sep 10 22:30:38.939442 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:38.939378 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:38.953666 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:38.953618 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" podStartSLOduration=1.953600894 podStartE2EDuration="1.953600894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:38.953371716 +0000 UTC m=+745.749587222" watchObservedRunningTime="2026-09-10 22:30:38.953600894 +0000 UTC m=+745.749816374" Sep 10 22:30:38.971913 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:38.971872 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" podStartSLOduration=2.677532246 podStartE2EDuration="2.971859227s" totalImagesPullingTime="294.326981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:38.969779363 +0000 UTC m=+745.765994841" watchObservedRunningTime="2026-09-10 22:30:38.971859227 +0000 UTC m=+745.768074703" Sep 10 22:30:44.950527 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:44.950493 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:30:49.947994 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:49.947953 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:30:55.633406 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:55.633371 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6"] Sep 10 22:30:55.633881 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:55.633614 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" podUID="e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" containerName="payload-processing" containerID="cri-o://a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1" gracePeriod=30 Sep 10 22:30:55.642901 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:55.642873 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k"] Sep 10 22:30:55.643121 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:55.643098 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" podUID="d366cde3-64af-4d2d-b2d4-f0fc3d984678" containerName="payload-pre-processing" containerID="cri-o://f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380" gracePeriod=30 Sep 10 22:30:55.914684 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:55.914658 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:55.941787 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:55.941760 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:56.032925 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.032894 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbwxw\" (UniqueName: \"kubernetes.io/projected/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-kube-api-access-dbwxw\") pod \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " Sep 10 22:30:56.033068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.032950 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcp56\" (UniqueName: \"kubernetes.io/projected/d366cde3-64af-4d2d-b2d4-f0fc3d984678-kube-api-access-qcp56\") pod \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " Sep 10 22:30:56.033068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.033039 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-plugins-config-volume\") pod \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\" (UID: \"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d\") " Sep 10 22:30:56.033068 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.033061 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d366cde3-64af-4d2d-b2d4-f0fc3d984678-plugins-config-volume\") pod \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\" (UID: \"d366cde3-64af-4d2d-b2d4-f0fc3d984678\") " Sep 10 22:30:56.033437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.033412 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" (UID: "e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:30:56.033656 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.033453 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d366cde3-64af-4d2d-b2d4-f0fc3d984678-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d366cde3-64af-4d2d-b2d4-f0fc3d984678" (UID: "d366cde3-64af-4d2d-b2d4-f0fc3d984678"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:30:56.035146 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.035122 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d366cde3-64af-4d2d-b2d4-f0fc3d984678-kube-api-access-qcp56" (OuterVolumeSpecName: "kube-api-access-qcp56") pod "d366cde3-64af-4d2d-b2d4-f0fc3d984678" (UID: "d366cde3-64af-4d2d-b2d4-f0fc3d984678"). InnerVolumeSpecName "kube-api-access-qcp56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:30:56.035249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.035170 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-kube-api-access-dbwxw" (OuterVolumeSpecName: "kube-api-access-dbwxw") pod "e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" (UID: "e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d"). InnerVolumeSpecName "kube-api-access-dbwxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:30:56.048274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.048242 2577 generic.go:356] "Generic (PLEG): container finished" podID="e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" containerID="a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1" exitCode=0 Sep 10 22:30:56.048365 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.048291 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" Sep 10 22:30:56.048365 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.048333 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" event={"ID":"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d","Type":"ContainerDied","Data":"a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1"} Sep 10 22:30:56.048449 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.048388 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6" event={"ID":"e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d","Type":"ContainerDied","Data":"3018052ddd3cfd7b6c75f194a0eda5e6481e8143a41ba00d8e0ff3c58ddf928f"} Sep 10 22:30:56.048449 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.048419 2577 scope.go:117] "RemoveContainer" containerID="a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1" Sep 10 22:30:56.050010 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.049984 2577 generic.go:356] "Generic (PLEG): container finished" podID="d366cde3-64af-4d2d-b2d4-f0fc3d984678" containerID="f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380" exitCode=0 Sep 10 22:30:56.050110 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.050017 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" event={"ID":"d366cde3-64af-4d2d-b2d4-f0fc3d984678","Type":"ContainerDied","Data":"f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380"} Sep 10 22:30:56.050110 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.050054 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" Sep 10 22:30:56.050110 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.050067 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k" event={"ID":"d366cde3-64af-4d2d-b2d4-f0fc3d984678","Type":"ContainerDied","Data":"f3cf1f48c0a8c253716de2cb5460162dbd85766352c0022c99b32156f72d2ff4"} Sep 10 22:30:56.061026 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.060924 2577 scope.go:117] "RemoveContainer" containerID="a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1" Sep 10 22:30:56.061293 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:30:56.061268 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1\": container with ID starting with a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1 not found: ID does not exist" containerID="a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1" Sep 10 22:30:56.061364 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.061302 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1"} err="failed to get container status \"a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1\": rpc error: code = NotFound desc = could not find container \"a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1\": container with ID starting with a5cc3903296f7ce805075ec3a8d9511cd5e931f5ec5aab5cfe1580e64a3db1d1 not found: ID does not exist" Sep 10 22:30:56.061364 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.061325 2577 scope.go:117] "RemoveContainer" containerID="f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380" Sep 10 22:30:56.072528 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.072439 2577 scope.go:117] "RemoveContainer" containerID="f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380" Sep 10 22:30:56.073481 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.073457 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6"] Sep 10 22:30:56.073726 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:30:56.073686 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380\": container with ID starting with f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380 not found: ID does not exist" containerID="f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380" Sep 10 22:30:56.073803 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.073721 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380"} err="failed to get container status \"f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380\": rpc error: code = NotFound desc = could not find container \"f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380\": container with ID starting with f7e3ce431a535ea2d0b10fc2079f3312099654a2fbce8443bc241c3adf73b380 not found: ID does not exist" Sep 10 22:30:56.082268 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.082249 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9005ccc0-59d58f86b9-27ss6"] Sep 10 22:30:56.092222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.092201 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k"] Sep 10 22:30:56.098027 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.098008 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9005ccc0-69969f598b-pkh9k"] Sep 10 22:30:56.133712 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.133689 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbwxw\" (UniqueName: \"kubernetes.io/projected/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-kube-api-access-dbwxw\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:30:56.133712 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.133709 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcp56\" (UniqueName: \"kubernetes.io/projected/d366cde3-64af-4d2d-b2d4-f0fc3d984678-kube-api-access-qcp56\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:30:56.133844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.133720 2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d-plugins-config-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:30:56.133844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.133729 2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d366cde3-64af-4d2d-b2d4-f0fc3d984678-plugins-config-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:30:56.923062 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.922988 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n"] Sep 10 22:30:56.923705 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.923685 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" containerName="payload-processing" Sep 10 22:30:56.923705 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.923705 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" containerName="payload-processing" Sep 10 22:30:56.923907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.923733 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d366cde3-64af-4d2d-b2d4-f0fc3d984678" containerName="payload-pre-processing" Sep 10 22:30:56.923907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.923742 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d366cde3-64af-4d2d-b2d4-f0fc3d984678" containerName="payload-pre-processing" Sep 10 22:30:56.923907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.923863 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" containerName="payload-processing" Sep 10 22:30:56.923907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.923882 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d366cde3-64af-4d2d-b2d4-f0fc3d984678" containerName="payload-pre-processing" Sep 10 22:30:56.929028 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.929005 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:56.931507 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.931489 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2643d53e-gw-openshift-default-dockercfg-8gh7g\"" Sep 10 22:30:56.938600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:56.938577 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n"] Sep 10 22:30:57.041478 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-data\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041622 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-istio-token\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041622 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041544 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-socket\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041622 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041598 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-certs\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041739 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-credential-socket\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041739 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041724 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85eae04b-01d5-43d1-a418-34ba131083c2-istiod-ca-cert\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041804 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85eae04b-01d5-43d1-a418-34ba131083c2-istio-podinfo\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041804 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041784 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-envoy\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.041898 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.041848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5lpd\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-kube-api-access-p5lpd\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142317 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-istio-token\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142474 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-socket\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142474 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142445 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-certs\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142546 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142524 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-credential-socket\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142609 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85eae04b-01d5-43d1-a418-34ba131083c2-istiod-ca-cert\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142703 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142685 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85eae04b-01d5-43d1-a418-34ba131083c2-istio-podinfo\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-envoy\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142741 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-socket\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142754 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-certs\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5lpd\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-kube-api-access-p5lpd\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-data\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.142991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.142895 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-credential-socket\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.143188 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.143168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-data\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.143265 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.143249 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85eae04b-01d5-43d1-a418-34ba131083c2-istiod-ca-cert\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.145173 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.145149 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-envoy\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.145281 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.145263 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85eae04b-01d5-43d1-a418-34ba131083c2-istio-podinfo\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.177738 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.177683 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-istio-token\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.177985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.177967 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5lpd\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-kube-api-access-p5lpd\") pod \"e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.241361 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.241333 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:30:57.522461 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.522430 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n"] Sep 10 22:30:57.523798 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:30:57.523772 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85eae04b_01d5_43d1_a418_34ba131083c2.slice/crio-5e0069b439765e1fb26c796293ce677b30b7bf75f95c792ade1200f48312d41f WatchSource:0}: Error finding container 5e0069b439765e1fb26c796293ce677b30b7bf75f95c792ade1200f48312d41f: Status 404 returned error can't find the container with id 5e0069b439765e1fb26c796293ce677b30b7bf75f95c792ade1200f48312d41f Sep 10 22:30:57.751126 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.751093 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d366cde3-64af-4d2d-b2d4-f0fc3d984678" path="/var/lib/kubelet/pods/d366cde3-64af-4d2d-b2d4-f0fc3d984678/volumes" Sep 10 22:30:57.751506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:57.751490 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d" path="/var/lib/kubelet/pods/e06b2ce7-b656-4cb3-b8a0-0ea096acfc1d/volumes" Sep 10 22:30:58.065989 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:30:58.065955 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" event={"ID":"85eae04b-01d5-43d1-a418-34ba131083c2","Type":"ContainerStarted","Data":"5e0069b439765e1fb26c796293ce677b30b7bf75f95c792ade1200f48312d41f"} Sep 10 22:31:00.084647 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:00.084611 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:31:00.085020 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:00.084691 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:31:00.085020 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:00.084724 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:31:01.089763 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:01.089723 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" event={"ID":"85eae04b-01d5-43d1-a418-34ba131083c2","Type":"ContainerStarted","Data":"03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c"} Sep 10 22:31:01.110646 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:01.110597 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" podStartSLOduration=2.552209697 podStartE2EDuration="5.110581197s" totalImagesPullingTime="2.5583715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:31:01.108917359 +0000 UTC m=+767.905132837" watchObservedRunningTime="2026-09-10 22:31:01.110581197 +0000 UTC m=+767.906796672" Sep 10 22:31:01.241469 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:01.241430 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:01.246707 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:01.246675 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:02.095785 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:02.095752 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:02.096796 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:02.096777 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:26.264854 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:26.264785 2577 generic.go:356] "Generic (PLEG): container finished" podID="85eae04b-01d5-43d1-a418-34ba131083c2" containerID="03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c" exitCode=0 Sep 10 22:31:26.264854 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:26.264840 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" event={"ID":"85eae04b-01d5-43d1-a418-34ba131083c2","Type":"ContainerDied","Data":"03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c"} Sep 10 22:31:26.265381 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:26.265192 2577 scope.go:117] "RemoveContainer" containerID="03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c" Sep 10 22:31:26.266403 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:26.266375 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:31:26.266469 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:26.266443 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:31:26.266505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:26.266473 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:31:27.273371 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:27.273340 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" event={"ID":"85eae04b-01d5-43d1-a418-34ba131083c2","Type":"ContainerStarted","Data":"da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3"} Sep 10 22:31:28.241643 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:28.241612 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:28.243099 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:28.243065 2577 patch_prober.go:28] interesting pod/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 10 22:31:28.243239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:28.243123 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 10 22:31:29.242054 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:29.242015 2577 patch_prober.go:28] interesting pod/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 10 22:31:29.242434 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:29.242073 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 10 22:31:30.242503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:30.242466 2577 patch_prober.go:28] interesting pod/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 10 22:31:30.242911 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:30.242524 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 10 22:31:31.242002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:31.241969 2577 patch_prober.go:28] interesting pod/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 10 22:31:31.242249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:31.242023 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 10 22:31:32.096526 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:32.096479 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:32.242234 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:32.242199 2577 patch_prober.go:28] interesting pod/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Sep 10 22:31:32.242401 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:32.242249 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Sep 10 22:31:33.246317 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:33.246290 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:33.247519 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:33.247493 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:33.908865 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:33.908835 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n"] Sep 10 22:31:34.334776 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:34.334717 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" containerID="cri-o://da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3" gracePeriod=30 Sep 10 22:31:39.584034 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.584007 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:39.763505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763438 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85eae04b-01d5-43d1-a418-34ba131083c2-istiod-ca-cert\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.763505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763498 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-credential-socket\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.763734 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763537 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85eae04b-01d5-43d1-a418-34ba131083c2-istio-podinfo\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.763734 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763590 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-certs\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.763734 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763633 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-socket\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.763734 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763656 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-istio-token\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.763734 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763725 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5lpd\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-kube-api-access-p5lpd\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.764031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763713 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:31:39.764031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763765 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-envoy\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.764031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763873 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:31:39.764031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763877 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-data\") pod \"85eae04b-01d5-43d1-a418-34ba131083c2\" (UID: \"85eae04b-01d5-43d1-a418-34ba131083c2\") " Sep 10 22:31:39.764031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763912 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:31:39.764031 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.763926 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85eae04b-01d5-43d1-a418-34ba131083c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:31:39.764308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.764074 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:31:39.764308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.764277 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.764308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.764292 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.764308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.764301 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.764308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.764309 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85eae04b-01d5-43d1-a418-34ba131083c2-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.764724 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.764321 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.766041 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.766009 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:31:39.766147 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.766087 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:31:39.766501 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.766481 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85eae04b-01d5-43d1-a418-34ba131083c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:31:39.766593 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.766573 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-kube-api-access-p5lpd" (OuterVolumeSpecName: "kube-api-access-p5lpd") pod "85eae04b-01d5-43d1-a418-34ba131083c2" (UID: "85eae04b-01d5-43d1-a418-34ba131083c2"). InnerVolumeSpecName "kube-api-access-p5lpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:31:39.864938 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.864907 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5lpd\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-kube-api-access-p5lpd\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.864938 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.864935 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85eae04b-01d5-43d1-a418-34ba131083c2-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.864938 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.864946 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85eae04b-01d5-43d1-a418-34ba131083c2-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:39.865154 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:39.864954 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85eae04b-01d5-43d1-a418-34ba131083c2-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:31:40.372371 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.372336 2577 generic.go:356] "Generic (PLEG): container finished" podID="85eae04b-01d5-43d1-a418-34ba131083c2" containerID="da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3" exitCode=0 Sep 10 22:31:40.372536 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.372380 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" Sep 10 22:31:40.372536 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.372416 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" event={"ID":"85eae04b-01d5-43d1-a418-34ba131083c2","Type":"ContainerDied","Data":"da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3"} Sep 10 22:31:40.372536 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.372470 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n" event={"ID":"85eae04b-01d5-43d1-a418-34ba131083c2","Type":"ContainerDied","Data":"5e0069b439765e1fb26c796293ce677b30b7bf75f95c792ade1200f48312d41f"} Sep 10 22:31:40.372536 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.372491 2577 scope.go:117] "RemoveContainer" containerID="da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3" Sep 10 22:31:40.383211 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.383183 2577 scope.go:117] "RemoveContainer" containerID="03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c" Sep 10 22:31:40.392937 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.392916 2577 scope.go:117] "RemoveContainer" containerID="da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3" Sep 10 22:31:40.393271 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:31:40.393249 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3\": container with ID starting with da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3 not found: ID does not exist" containerID="da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3" Sep 10 22:31:40.393383 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.393277 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3"} err="failed to get container status \"da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3\": rpc error: code = NotFound desc = could not find container \"da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3\": container with ID starting with da6e686e0c3bdbeba64024e1599ed4545df15cbbe30acfd2a50063de421695e3 not found: ID does not exist" Sep 10 22:31:40.393383 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.393304 2577 scope.go:117] "RemoveContainer" containerID="03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c" Sep 10 22:31:40.393604 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:31:40.393587 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c\": container with ID starting with 03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c not found: ID does not exist" containerID="03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c" Sep 10 22:31:40.393686 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.393613 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c"} err="failed to get container status \"03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c\": rpc error: code = NotFound desc = could not find container \"03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c\": container with ID starting with 03348ff0ea2a6b13237e97681b1455472b4259227fc9a45171cf39c57bca3f5c not found: ID does not exist" Sep 10 22:31:40.394501 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.394480 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n"] Sep 10 22:31:40.406672 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:40.406649 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2643d53e-gw-openshift-default-6cd6bb4b56-fsh8n"] Sep 10 22:31:41.750996 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:31:41.750962 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" path="/var/lib/kubelet/pods/85eae04b-01d5-43d1-a418-34ba131083c2/volumes" Sep 10 22:32:10.862683 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.862649 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn"] Sep 10 22:32:10.863424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.863404 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" Sep 10 22:32:10.863472 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.863428 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" Sep 10 22:32:10.863472 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.863452 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" Sep 10 22:32:10.863472 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.863460 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" Sep 10 22:32:10.863574 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.863560 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" Sep 10 22:32:10.863628 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.863583 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="85eae04b-01d5-43d1-a418-34ba131083c2" containerName="istio-proxy" Sep 10 22:32:10.866989 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.866970 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.869103 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.869081 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-e01e31eb-openshift-default-dockercfg-bzrrw\"" Sep 10 22:32:10.877340 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.877316 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn"] Sep 10 22:32:10.945274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-credential-socket\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945447 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945297 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-token\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945447 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945328 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-envoy\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945447 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945356 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj8h6\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-kube-api-access-jj8h6\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945447 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945419 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-socket\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-certs\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-podinfo\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945551 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istiod-ca-cert\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:10.945714 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:10.945609 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-data\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046084 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-podinfo\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046084 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istiod-ca-cert\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046129 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-data\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-credential-socket\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-token\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046201 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-envoy\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046227 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj8h6\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-kube-api-access-jj8h6\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046257 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-socket\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-certs\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046674 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046583 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-data\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046775 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-credential-socket\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046869 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046803 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-certs\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.046927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.046888 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-socket\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.047165 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.047142 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istiod-ca-cert\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.048630 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.048607 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-envoy\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.048864 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.048844 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-podinfo\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.054079 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.054053 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-token\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.054511 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.054487 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj8h6\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-kube-api-access-jj8h6\") pod \"e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.180617 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.180531 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:11.478517 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.478494 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn"] Sep 10 22:32:11.480184 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:32:11.480160 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a74a9dd_b5a7_4755_a7db_00a24d074d29.slice/crio-80d63fbaf8b63f19174d838f253c629b3b211e2df411d86d51bac174f98315e3 WatchSource:0}: Error finding container 80d63fbaf8b63f19174d838f253c629b3b211e2df411d86d51bac174f98315e3: Status 404 returned error can't find the container with id 80d63fbaf8b63f19174d838f253c629b3b211e2df411d86d51bac174f98315e3 Sep 10 22:32:11.482503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.482474 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:32:11.482556 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.482538 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:32:11.482596 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.482565 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:32:11.569781 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.569744 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" event={"ID":"3a74a9dd-b5a7-4755-a7db-00a24d074d29","Type":"ContainerStarted","Data":"044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a"} Sep 10 22:32:11.569953 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.569829 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" event={"ID":"3a74a9dd-b5a7-4755-a7db-00a24d074d29","Type":"ContainerStarted","Data":"80d63fbaf8b63f19174d838f253c629b3b211e2df411d86d51bac174f98315e3"} Sep 10 22:32:11.590850 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:11.590774 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" podStartSLOduration=1.590754044 podStartE2EDuration="1.590754044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:32:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:32:11.588860657 +0000 UTC m=+838.385076135" watchObservedRunningTime="2026-09-10 22:32:11.590754044 +0000 UTC m=+838.386969520" Sep 10 22:32:12.181119 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:12.181090 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:12.187602 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:12.187572 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:12.576613 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:12.576583 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:12.577591 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:12.577571 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:12.676029 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:12.676002 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd"] Sep 10 22:32:12.676221 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:12.676198 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" podUID="7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" containerName="payload-pre-processing" containerID="cri-o://30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129" gracePeriod=30 Sep 10 22:32:12.958549 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:12.958523 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:32:13.068269 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.068232 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-plugins-config-volume\") pod \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " Sep 10 22:32:13.068482 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.068325 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjnjt\" (UniqueName: \"kubernetes.io/projected/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-kube-api-access-cjnjt\") pod \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\" (UID: \"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf\") " Sep 10 22:32:13.068704 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.068645 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" (UID: "7dfdb48a-069d-4b68-88e6-74ba6a3eecbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:32:13.071502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.071470 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-kube-api-access-cjnjt" (OuterVolumeSpecName: "kube-api-access-cjnjt") pod "7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" (UID: "7dfdb48a-069d-4b68-88e6-74ba6a3eecbf"). InnerVolumeSpecName "kube-api-access-cjnjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:32:13.169281 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.169207 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjnjt\" (UniqueName: \"kubernetes.io/projected/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-kube-api-access-cjnjt\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:13.169281 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.169237 2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf-plugins-config-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:13.583667 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.583636 2577 generic.go:356] "Generic (PLEG): container finished" podID="7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" containerID="30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129" exitCode=0 Sep 10 22:32:13.584094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.583673 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" Sep 10 22:32:13.584094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.583711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" event={"ID":"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf","Type":"ContainerDied","Data":"30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129"} Sep 10 22:32:13.584094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.583760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd" event={"ID":"7dfdb48a-069d-4b68-88e6-74ba6a3eecbf","Type":"ContainerDied","Data":"46df46e6af5a19608bab65e816ef6a66f2404426f131267831728dd9853517ed"} Sep 10 22:32:13.584094 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.583781 2577 scope.go:117] "RemoveContainer" containerID="30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129" Sep 10 22:32:13.594575 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.594558 2577 scope.go:117] "RemoveContainer" containerID="30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129" Sep 10 22:32:13.594889 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:32:13.594858 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129\": container with ID starting with 30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129 not found: ID does not exist" containerID="30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129" Sep 10 22:32:13.594959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.594901 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129"} err="failed to get container status \"30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129\": rpc error: code = NotFound desc = could not find container \"30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129\": container with ID starting with 30749a0a33743912cf07e15504905587b7296cc739bd50d766585a87d5018129 not found: ID does not exist" Sep 10 22:32:13.606972 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.606944 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd"] Sep 10 22:32:13.609320 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.609302 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-cc3245-d75d48d9d-g65cd"] Sep 10 22:32:13.759852 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:13.759805 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" path="/var/lib/kubelet/pods/7dfdb48a-069d-4b68-88e6-74ba6a3eecbf/volumes" Sep 10 22:32:37.742871 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:37.742839 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerID="044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a" exitCode=0 Sep 10 22:32:37.749800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:37.749767 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" event={"ID":"3a74a9dd-b5a7-4755-a7db-00a24d074d29","Type":"ContainerDied","Data":"044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a"} Sep 10 22:32:37.750209 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:37.750192 2577 scope.go:117] "RemoveContainer" containerID="044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a" Sep 10 22:32:37.751293 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:37.751257 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:32:37.751445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:37.751321 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:32:37.751445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:37.751353 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:32:38.750979 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:38.750942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" event={"ID":"3a74a9dd-b5a7-4755-a7db-00a24d074d29","Type":"ContainerStarted","Data":"f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3"} Sep 10 22:32:39.181772 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:39.181729 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:39.183064 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:39.183041 2577 patch_prober.go:28] interesting pod/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Sep 10 22:32:39.183169 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:39.183102 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Sep 10 22:32:40.181309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:40.181275 2577 patch_prober.go:28] interesting pod/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Sep 10 22:32:40.181682 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:40.181330 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Sep 10 22:32:41.181329 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:41.181296 2577 patch_prober.go:28] interesting pod/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Sep 10 22:32:41.181738 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:41.181349 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Sep 10 22:32:42.181770 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:42.181735 2577 patch_prober.go:28] interesting pod/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Sep 10 22:32:42.182176 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:42.181791 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Sep 10 22:32:42.576522 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:42.576490 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:43.181524 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:43.181486 2577 patch_prober.go:28] interesting pod/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Sep 10 22:32:43.181691 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:43.181570 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Sep 10 22:32:44.185527 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:44.185497 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:44.187185 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:44.187163 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:45.913058 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:45.913025 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn"] Sep 10 22:32:45.913433 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:45.913275 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" containerID="cri-o://f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3" gracePeriod=30 Sep 10 22:32:51.173076 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.173050 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:51.235628 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235594 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-envoy\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.235799 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235687 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj8h6\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-kube-api-access-jj8h6\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.235799 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235712 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-data\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.235799 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235755 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-token\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.235799 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235784 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-certs\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.236058 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235811 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-socket\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.236058 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235872 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-podinfo\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.236058 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235909 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-credential-socket\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.236058 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.235937 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istiod-ca-cert\") pod \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\" (UID: \"3a74a9dd-b5a7-4755-a7db-00a24d074d29\") " Sep 10 22:32:51.236292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.236047 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:32:51.236292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.236108 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:32:51.236292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.236278 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:32:51.236456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.236286 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:32:51.236456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.236345 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.236456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.236364 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.236456 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.236371 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:32:51.238239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.238214 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:32:51.238437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.238412 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:32:51.238524 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.238506 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:32:51.238524 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.238506 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-kube-api-access-jj8h6" (OuterVolumeSpecName: "kube-api-access-jj8h6") pod "3a74a9dd-b5a7-4755-a7db-00a24d074d29" (UID: "3a74a9dd-b5a7-4755-a7db-00a24d074d29"). InnerVolumeSpecName "kube-api-access-jj8h6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:32:51.337832 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.337788 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.337832 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.337812 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj8h6\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-kube-api-access-jj8h6\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.337832 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.337836 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.338085 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.337846 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.338085 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.337855 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.338085 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.337863 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a74a9dd-b5a7-4755-a7db-00a24d074d29-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.338085 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.337872 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a74a9dd-b5a7-4755-a7db-00a24d074d29-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:32:51.827868 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.827839 2577 generic.go:356] "Generic (PLEG): container finished" podID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerID="f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3" exitCode=0 Sep 10 22:32:51.828023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.827873 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" Sep 10 22:32:51.828023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.827892 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" event={"ID":"3a74a9dd-b5a7-4755-a7db-00a24d074d29","Type":"ContainerDied","Data":"f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3"} Sep 10 22:32:51.828023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.827932 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn" event={"ID":"3a74a9dd-b5a7-4755-a7db-00a24d074d29","Type":"ContainerDied","Data":"80d63fbaf8b63f19174d838f253c629b3b211e2df411d86d51bac174f98315e3"} Sep 10 22:32:51.828023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.827955 2577 scope.go:117] "RemoveContainer" containerID="f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3" Sep 10 22:32:51.837338 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.837303 2577 scope.go:117] "RemoveContainer" containerID="044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a" Sep 10 22:32:51.846931 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.846905 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn"] Sep 10 22:32:51.847714 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.847697 2577 scope.go:117] "RemoveContainer" containerID="f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3" Sep 10 22:32:51.848020 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:32:51.848002 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3\": container with ID starting with f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3 not found: ID does not exist" containerID="f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3" Sep 10 22:32:51.848083 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.848030 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3"} err="failed to get container status \"f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3\": rpc error: code = NotFound desc = could not find container \"f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3\": container with ID starting with f3783e60db0fe324da0a4d33016b42449d3e4f5548c62123b021cb4aca9304d3 not found: ID does not exist" Sep 10 22:32:51.848083 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.848046 2577 scope.go:117] "RemoveContainer" containerID="044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a" Sep 10 22:32:51.848274 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:32:51.848258 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a\": container with ID starting with 044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a not found: ID does not exist" containerID="044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a" Sep 10 22:32:51.848323 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.848277 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a"} err="failed to get container status \"044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a\": rpc error: code = NotFound desc = could not find container \"044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a\": container with ID starting with 044dc9edd6fd47e46cfc49d09329a24ab66d0a906ce9a92c2cd785fe460e3c9a not found: ID does not exist" Sep 10 22:32:51.849786 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:51.849767 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-e01e31eb-openshift-default-c55d9ccd8-dkcwn"] Sep 10 22:32:53.756778 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:32:53.756743 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" path="/var/lib/kubelet/pods/3a74a9dd-b5a7-4755-a7db-00a24d074d29/volumes" Sep 10 22:33:13.707498 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:33:13.707463 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:33:13.709228 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:33:13.709208 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:34:42.791659 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:42.791576 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9"] Sep 10 22:34:42.792427 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:42.792373 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" podUID="eddebc97-2248-457a-81d4-f7048b1372c6" containerName="maas-api" containerID="cri-o://d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac" gracePeriod=30 Sep 10 22:34:42.856890 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:42.856805 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4"] Sep 10 22:34:42.857203 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:42.857149 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" podUID="0b1331ab-b40b-47c2-8098-0f0cdd09858d" containerName="payload-processing" containerID="cri-o://3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29" gracePeriod=30 Sep 10 22:34:43.104597 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.104570 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:34:43.142927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.142895 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:34:43.159114 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.159081 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lggf\" (UniqueName: \"kubernetes.io/projected/eddebc97-2248-457a-81d4-f7048b1372c6-kube-api-access-5lggf\") pod \"eddebc97-2248-457a-81d4-f7048b1372c6\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " Sep 10 22:34:43.159251 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.159161 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-maas-api-tls\") pod \"eddebc97-2248-457a-81d4-f7048b1372c6\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " Sep 10 22:34:43.159251 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.159187 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-metrics-certs\") pod \"eddebc97-2248-457a-81d4-f7048b1372c6\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " Sep 10 22:34:43.159251 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.159243 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jftdm\" (UniqueName: \"kubernetes.io/projected/0b1331ab-b40b-47c2-8098-0f0cdd09858d-kube-api-access-jftdm\") pod \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " Sep 10 22:34:43.159430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.159298 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b1331ab-b40b-47c2-8098-0f0cdd09858d-plugins-config-volume\") pod \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\" (UID: \"0b1331ab-b40b-47c2-8098-0f0cdd09858d\") " Sep 10 22:34:43.159430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.159325 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eddebc97-2248-457a-81d4-f7048b1372c6-openshift-service-ca\") pod \"eddebc97-2248-457a-81d4-f7048b1372c6\" (UID: \"eddebc97-2248-457a-81d4-f7048b1372c6\") " Sep 10 22:34:43.159967 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.159940 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eddebc97-2248-457a-81d4-f7048b1372c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eddebc97-2248-457a-81d4-f7048b1372c6" (UID: "eddebc97-2248-457a-81d4-f7048b1372c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:34:43.160269 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.160227 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b1331ab-b40b-47c2-8098-0f0cdd09858d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b1331ab-b40b-47c2-8098-0f0cdd09858d" (UID: "0b1331ab-b40b-47c2-8098-0f0cdd09858d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:34:43.162129 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.162104 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eddebc97-2248-457a-81d4-f7048b1372c6" (UID: "eddebc97-2248-457a-81d4-f7048b1372c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:34:43.162293 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.162269 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eddebc97-2248-457a-81d4-f7048b1372c6-kube-api-access-5lggf" (OuterVolumeSpecName: "kube-api-access-5lggf") pod "eddebc97-2248-457a-81d4-f7048b1372c6" (UID: "eddebc97-2248-457a-81d4-f7048b1372c6"). InnerVolumeSpecName "kube-api-access-5lggf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:34:43.162379 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.162268 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eddebc97-2248-457a-81d4-f7048b1372c6" (UID: "eddebc97-2248-457a-81d4-f7048b1372c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:34:43.163745 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.163718 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b1331ab-b40b-47c2-8098-0f0cdd09858d-kube-api-access-jftdm" (OuterVolumeSpecName: "kube-api-access-jftdm") pod "0b1331ab-b40b-47c2-8098-0f0cdd09858d" (UID: "0b1331ab-b40b-47c2-8098-0f0cdd09858d"). InnerVolumeSpecName "kube-api-access-jftdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:34:43.260227 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.260196 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-maas-api-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:34:43.260227 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.260223 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eddebc97-2248-457a-81d4-f7048b1372c6-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:34:43.260227 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.260235 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jftdm\" (UniqueName: \"kubernetes.io/projected/0b1331ab-b40b-47c2-8098-0f0cdd09858d-kube-api-access-jftdm\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:34:43.260448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.260244 2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b1331ab-b40b-47c2-8098-0f0cdd09858d-plugins-config-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:34:43.260448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.260253 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eddebc97-2248-457a-81d4-f7048b1372c6-openshift-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:34:43.260448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.260262 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lggf\" (UniqueName: \"kubernetes.io/projected/eddebc97-2248-457a-81d4-f7048b1372c6-kube-api-access-5lggf\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:34:43.507745 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.507658 2577 generic.go:356] "Generic (PLEG): container finished" podID="eddebc97-2248-457a-81d4-f7048b1372c6" containerID="d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac" exitCode=0 Sep 10 22:34:43.507745 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.507706 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" Sep 10 22:34:43.507946 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.507744 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" event={"ID":"eddebc97-2248-457a-81d4-f7048b1372c6","Type":"ContainerDied","Data":"d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac"} Sep 10 22:34:43.507946 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.507808 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9" event={"ID":"eddebc97-2248-457a-81d4-f7048b1372c6","Type":"ContainerDied","Data":"7152b89073b43f01affa1ad56a45a50673cc498838925b4b1ec80eb66fc3a04d"} Sep 10 22:34:43.507946 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.507854 2577 scope.go:117] "RemoveContainer" containerID="d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac" Sep 10 22:34:43.509371 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.509346 2577 generic.go:356] "Generic (PLEG): container finished" podID="0b1331ab-b40b-47c2-8098-0f0cdd09858d" containerID="3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29" exitCode=0 Sep 10 22:34:43.509494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.509385 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" Sep 10 22:34:43.509494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.509382 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" event={"ID":"0b1331ab-b40b-47c2-8098-0f0cdd09858d","Type":"ContainerDied","Data":"3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29"} Sep 10 22:34:43.509608 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.509531 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4" event={"ID":"0b1331ab-b40b-47c2-8098-0f0cdd09858d","Type":"ContainerDied","Data":"20eb0f76b9590b42f98082b9e5dd151ef57870a92bec74df416ace0228fcfb73"} Sep 10 22:34:43.517225 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.517209 2577 scope.go:117] "RemoveContainer" containerID="d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac" Sep 10 22:34:43.517520 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:34:43.517493 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac\": container with ID starting with d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac not found: ID does not exist" containerID="d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac" Sep 10 22:34:43.517611 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.517532 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac"} err="failed to get container status \"d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac\": rpc error: code = NotFound desc = could not find container \"d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac\": container with ID starting with d017c6cd2f8fcd6547767dc2e06297057d4cdd86a238cf8a7a97c4e6b1b7d5ac not found: ID does not exist" Sep 10 22:34:43.517611 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.517554 2577 scope.go:117] "RemoveContainer" containerID="3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29" Sep 10 22:34:43.526623 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.526606 2577 scope.go:117] "RemoveContainer" containerID="3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29" Sep 10 22:34:43.526899 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:34:43.526880 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29\": container with ID starting with 3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29 not found: ID does not exist" containerID="3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29" Sep 10 22:34:43.526946 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.526905 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29"} err="failed to get container status \"3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29\": rpc error: code = NotFound desc = could not find container \"3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29\": container with ID starting with 3de03896235830de1008ab0c9759a929a681dfdfd736e270d2b11d87c45fad29 not found: ID does not exist" Sep 10 22:34:43.534436 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.534412 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9"] Sep 10 22:34:43.539048 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.539029 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ddf324-97dbcbd95-k2fg9"] Sep 10 22:34:43.556246 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.556227 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4"] Sep 10 22:34:43.559853 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.559833 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-ddf324-7c9c4bc95-kgwr4"] Sep 10 22:34:43.751357 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.751317 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b1331ab-b40b-47c2-8098-0f0cdd09858d" path="/var/lib/kubelet/pods/0b1331ab-b40b-47c2-8098-0f0cdd09858d/volumes" Sep 10 22:34:43.752016 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:34:43.751984 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eddebc97-2248-457a-81d4-f7048b1372c6" path="/var/lib/kubelet/pods/eddebc97-2248-457a-81d4-f7048b1372c6/volumes" Sep 10 22:35:10.176328 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.176289 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j"] Sep 10 22:35:10.176893 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.176544 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" podUID="4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" containerName="maas-api" containerID="cri-o://f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b" gracePeriod=30 Sep 10 22:35:10.246517 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.246482 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4"] Sep 10 22:35:10.246942 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.246726 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" podUID="01390529-63ad-4e84-a073-e29cc04b3192" containerName="payload-pre-processing" containerID="cri-o://abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6" gracePeriod=30 Sep 10 22:35:10.503437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.503408 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:35:10.540976 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.540948 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:35:10.617486 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.617449 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01390529-63ad-4e84-a073-e29cc04b3192-plugins-config-volume\") pod \"01390529-63ad-4e84-a073-e29cc04b3192\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " Sep 10 22:35:10.617486 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.617490 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckdkg\" (UniqueName: \"kubernetes.io/projected/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-kube-api-access-ckdkg\") pod \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " Sep 10 22:35:10.617744 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.617509 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-metrics-certs\") pod \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " Sep 10 22:35:10.617744 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.617712 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls\") pod \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " Sep 10 22:35:10.617911 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.617879 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jccjd\" (UniqueName: \"kubernetes.io/projected/01390529-63ad-4e84-a073-e29cc04b3192-kube-api-access-jccjd\") pod \"01390529-63ad-4e84-a073-e29cc04b3192\" (UID: \"01390529-63ad-4e84-a073-e29cc04b3192\") " Sep 10 22:35:10.617971 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.617916 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-openshift-service-ca\") pod \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\" (UID: \"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12\") " Sep 10 22:35:10.617971 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.617951 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01390529-63ad-4e84-a073-e29cc04b3192-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "01390529-63ad-4e84-a073-e29cc04b3192" (UID: "01390529-63ad-4e84-a073-e29cc04b3192"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:35:10.618242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.618224 2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01390529-63ad-4e84-a073-e29cc04b3192-plugins-config-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:35:10.618296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.618287 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" (UID: "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:35:10.619973 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.619946 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" (UID: "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:35:10.620248 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.620229 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01390529-63ad-4e84-a073-e29cc04b3192-kube-api-access-jccjd" (OuterVolumeSpecName: "kube-api-access-jccjd") pod "01390529-63ad-4e84-a073-e29cc04b3192" (UID: "01390529-63ad-4e84-a073-e29cc04b3192"). InnerVolumeSpecName "kube-api-access-jccjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:35:10.620349 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.620333 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" (UID: "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:35:10.620411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.620368 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-kube-api-access-ckdkg" (OuterVolumeSpecName: "kube-api-access-ckdkg") pod "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" (UID: "4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12"). InnerVolumeSpecName "kube-api-access-ckdkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:35:10.672155 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.672120 2577 generic.go:356] "Generic (PLEG): container finished" podID="01390529-63ad-4e84-a073-e29cc04b3192" containerID="abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6" exitCode=0 Sep 10 22:35:10.672296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.672160 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" Sep 10 22:35:10.672296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.672205 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" event={"ID":"01390529-63ad-4e84-a073-e29cc04b3192","Type":"ContainerDied","Data":"abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6"} Sep 10 22:35:10.672296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.672256 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4" event={"ID":"01390529-63ad-4e84-a073-e29cc04b3192","Type":"ContainerDied","Data":"b78b75dd7bf8aab1fb793fc84571708741e14285143eb9810daaed0dfe8ce570"} Sep 10 22:35:10.672296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.672280 2577 scope.go:117] "RemoveContainer" containerID="abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6" Sep 10 22:35:10.674299 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.674279 2577 generic.go:356] "Generic (PLEG): container finished" podID="4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" containerID="f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b" exitCode=0 Sep 10 22:35:10.674394 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.674307 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" event={"ID":"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12","Type":"ContainerDied","Data":"f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b"} Sep 10 22:35:10.674394 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.674330 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" Sep 10 22:35:10.674394 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.674337 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j" event={"ID":"4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12","Type":"ContainerDied","Data":"55bfc1ac9f91265170016c8b93f978480355874b03d3b72f8a32f1a2d0d246d9"} Sep 10 22:35:10.685895 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.685875 2577 scope.go:117] "RemoveContainer" containerID="abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6" Sep 10 22:35:10.687354 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:35:10.687330 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6\": container with ID starting with abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6 not found: ID does not exist" containerID="abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6" Sep 10 22:35:10.687473 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.687387 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6"} err="failed to get container status \"abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6\": rpc error: code = NotFound desc = could not find container \"abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6\": container with ID starting with abd4d9e80604bb992f3ac13f53927da2a91d0f557b5eac840359c9c6d031e0f6 not found: ID does not exist" Sep 10 22:35:10.687473 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.687413 2577 scope.go:117] "RemoveContainer" containerID="f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b" Sep 10 22:35:10.698424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.697941 2577 scope.go:117] "RemoveContainer" containerID="f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b" Sep 10 22:35:10.698424 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:35:10.698224 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b\": container with ID starting with f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b not found: ID does not exist" containerID="f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b" Sep 10 22:35:10.698424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.698248 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b"} err="failed to get container status \"f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b\": rpc error: code = NotFound desc = could not find container \"f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b\": container with ID starting with f207d9594a00ae1167cdbf6acf631ce4d565b1f6bc85c4b54d07966b1cb0c86b not found: ID does not exist" Sep 10 22:35:10.708001 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.707950 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4"] Sep 10 22:35:10.711394 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.711370 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d1a134-6cdb6b7f58-bxrh4"] Sep 10 22:35:10.718854 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.718829 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jccjd\" (UniqueName: \"kubernetes.io/projected/01390529-63ad-4e84-a073-e29cc04b3192-kube-api-access-jccjd\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:35:10.718947 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.718860 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-openshift-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:35:10.718947 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.718875 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckdkg\" (UniqueName: \"kubernetes.io/projected/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-kube-api-access-ckdkg\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:35:10.718947 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.718888 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:35:10.718947 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.718900 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12-maas-api-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:35:10.721302 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.721281 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j"] Sep 10 22:35:10.725809 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:10.725790 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d1a134-5fbbc6696c-bkv4j"] Sep 10 22:35:11.750192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.750162 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01390529-63ad-4e84-a073-e29cc04b3192" path="/var/lib/kubelet/pods/01390529-63ad-4e84-a073-e29cc04b3192/volumes" Sep 10 22:35:11.750604 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.750586 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" path="/var/lib/kubelet/pods/4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12/volumes" Sep 10 22:35:11.980345 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980314 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd"] Sep 10 22:35:11.980711 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980698 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eddebc97-2248-457a-81d4-f7048b1372c6" containerName="maas-api" Sep 10 22:35:11.980756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980713 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eddebc97-2248-457a-81d4-f7048b1372c6" containerName="maas-api" Sep 10 22:35:11.980756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980726 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" containerName="maas-api" Sep 10 22:35:11.980756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980732 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" containerName="maas-api" Sep 10 22:35:11.980756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980738 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" Sep 10 22:35:11.980756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980744 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980759 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b1331ab-b40b-47c2-8098-0f0cdd09858d" containerName="payload-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980764 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1331ab-b40b-47c2-8098-0f0cdd09858d" containerName="payload-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980771 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" containerName="payload-pre-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980776 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" containerName="payload-pre-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980785 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01390529-63ad-4e84-a073-e29cc04b3192" containerName="payload-pre-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980790 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="01390529-63ad-4e84-a073-e29cc04b3192" containerName="payload-pre-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980867 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980877 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b1331ab-b40b-47c2-8098-0f0cdd09858d" containerName="payload-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980886 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e90ce12-c4b1-4ab9-83a7-2c1e7e6cca12" containerName="maas-api" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980892 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eddebc97-2248-457a-81d4-f7048b1372c6" containerName="maas-api" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980897 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dfdb48a-069d-4b68-88e6-74ba6a3eecbf" containerName="payload-pre-processing" Sep 10 22:35:11.980927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.980905 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="01390529-63ad-4e84-a073-e29cc04b3192" containerName="payload-pre-processing" Sep 10 22:35:11.985324 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.985303 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" Sep 10 22:35:11.988843 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.988785 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bltkc\"" Sep 10 22:35:11.990467 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:11.990444 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd"] Sep 10 22:35:12.132052 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:12.132021 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35ac677-9efa-4999-8236-5b4b5b402a63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b902065f-r4lzd\" (UID: \"f35ac677-9efa-4999-8236-5b4b5b402a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" Sep 10 22:35:12.233389 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:12.233358 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35ac677-9efa-4999-8236-5b4b5b402a63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b902065f-r4lzd\" (UID: \"f35ac677-9efa-4999-8236-5b4b5b402a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" Sep 10 22:35:12.233974 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:12.233956 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35ac677-9efa-4999-8236-5b4b5b402a63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b902065f-r4lzd\" (UID: \"f35ac677-9efa-4999-8236-5b4b5b402a63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" Sep 10 22:35:12.297559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:12.297529 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" Sep 10 22:35:12.570296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:12.570274 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd"] Sep 10 22:35:12.572628 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:35:12.572589 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35ac677_9efa_4999_8236_5b4b5b402a63.slice/crio-cb01d4be905be65c482a8bffb7697080a20c4fd833810113dfa5dcad6ac20caa WatchSource:0}: Error finding container cb01d4be905be65c482a8bffb7697080a20c4fd833810113dfa5dcad6ac20caa: Status 404 returned error can't find the container with id cb01d4be905be65c482a8bffb7697080a20c4fd833810113dfa5dcad6ac20caa Sep 10 22:35:12.689209 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:12.689174 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" event={"ID":"f35ac677-9efa-4999-8236-5b4b5b402a63","Type":"ContainerStarted","Data":"cb01d4be905be65c482a8bffb7697080a20c4fd833810113dfa5dcad6ac20caa"} Sep 10 22:35:13.696644 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:13.696550 2577 generic.go:356] "Generic (PLEG): container finished" podID="f35ac677-9efa-4999-8236-5b4b5b402a63" containerID="6513fe30c3410a49f46ca6a24ef6e27a44dce61281c7b36f60373e9b5643f789" exitCode=0 Sep 10 22:35:13.697061 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:13.696647 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" event={"ID":"f35ac677-9efa-4999-8236-5b4b5b402a63","Type":"ContainerDied","Data":"6513fe30c3410a49f46ca6a24ef6e27a44dce61281c7b36f60373e9b5643f789"} Sep 10 22:35:14.832000 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:14.831969 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" Sep 10 22:35:14.960166 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:14.960085 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35ac677-9efa-4999-8236-5b4b5b402a63-maas-api-service-ca\") pod \"f35ac677-9efa-4999-8236-5b4b5b402a63\" (UID: \"f35ac677-9efa-4999-8236-5b4b5b402a63\") " Sep 10 22:35:14.960405 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:14.960382 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f35ac677-9efa-4999-8236-5b4b5b402a63-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f35ac677-9efa-4999-8236-5b4b5b402a63" (UID: "f35ac677-9efa-4999-8236-5b4b5b402a63"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:35:15.061617 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:15.061587 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f35ac677-9efa-4999-8236-5b4b5b402a63-maas-api-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:35:15.710212 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:15.710175 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" event={"ID":"f35ac677-9efa-4999-8236-5b4b5b402a63","Type":"ContainerDied","Data":"cb01d4be905be65c482a8bffb7697080a20c4fd833810113dfa5dcad6ac20caa"} Sep 10 22:35:15.710212 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:15.710215 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd" Sep 10 22:35:15.710423 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:15.710224 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb01d4be905be65c482a8bffb7697080a20c4fd833810113dfa5dcad6ac20caa" Sep 10 22:35:17.913082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.913045 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt"] Sep 10 22:35:17.913696 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.913674 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35ac677-9efa-4999-8236-5b4b5b402a63" containerName="revoke-keys" Sep 10 22:35:17.913760 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.913700 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35ac677-9efa-4999-8236-5b4b5b402a63" containerName="revoke-keys" Sep 10 22:35:17.913760 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.913715 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" Sep 10 22:35:17.913760 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.913724 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" Sep 10 22:35:17.913947 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.913893 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35ac677-9efa-4999-8236-5b4b5b402a63" containerName="revoke-keys" Sep 10 22:35:17.913947 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.913914 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a74a9dd-b5a7-4755-a7db-00a24d074d29" containerName="istio-proxy" Sep 10 22:35:17.917806 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.917783 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:17.919951 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.919925 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-58cd39-openshift-default-dockercfg-9c5vr\"" Sep 10 22:35:17.930023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:17.929997 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt"] Sep 10 22:35:18.089116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-token\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089118 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-certs\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-data\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089223 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqjdm\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-kube-api-access-qqjdm\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istiod-ca-cert\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089322 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-credential-socket\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089454 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089346 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-socket\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089454 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089368 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-envoy\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.089454 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.089406 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-podinfo\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.190594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.190499 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-token\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.190594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.190543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-certs\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.190842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.190674 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-data\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.190842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.190759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqjdm\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-kube-api-access-qqjdm\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.190842 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.190797 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istiod-ca-cert\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191012 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.190947 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-certs\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191012 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.190952 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-credential-socket\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191118 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.191050 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-socket\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191118 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.191084 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-data\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191118 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.191087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-envoy\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.191158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-podinfo\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.191217 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-credential-socket\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191498 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.191474 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-socket\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.191596 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.191506 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istiod-ca-cert\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.193757 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.193735 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-envoy\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.194036 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.194019 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-podinfo\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.201767 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.201595 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-token\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.202286 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.202264 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqjdm\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-kube-api-access-qqjdm\") pod \"e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.230703 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.230681 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:18.512450 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.512424 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt"] Sep 10 22:35:18.513858 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:35:18.513801 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eded38c_eb50_406d_8f02_8cdf3c4e83c0.slice/crio-64b868e42a2438085270c70739a1851b6bcddc9accd96d0bbf60710327617627 WatchSource:0}: Error finding container 64b868e42a2438085270c70739a1851b6bcddc9accd96d0bbf60710327617627: Status 404 returned error can't find the container with id 64b868e42a2438085270c70739a1851b6bcddc9accd96d0bbf60710327617627 Sep 10 22:35:18.516213 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.516179 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:35:18.516299 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.516244 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:35:18.516299 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.516274 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:35:18.730586 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.730555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" event={"ID":"3eded38c-eb50-406d-8f02-8cdf3c4e83c0","Type":"ContainerStarted","Data":"b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4"} Sep 10 22:35:18.730760 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.730617 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" event={"ID":"3eded38c-eb50-406d-8f02-8cdf3c4e83c0","Type":"ContainerStarted","Data":"64b868e42a2438085270c70739a1851b6bcddc9accd96d0bbf60710327617627"} Sep 10 22:35:18.750238 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:18.750184 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podStartSLOduration=1.750169348 podStartE2EDuration="1.750169348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:35:18.747980943 +0000 UTC m=+1025.544196431" watchObservedRunningTime="2026-09-10 22:35:18.750169348 +0000 UTC m=+1025.546384825" Sep 10 22:35:19.230851 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:19.230812 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:19.236122 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:19.236095 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:19.737474 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:19.737440 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:19.738503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:19.738479 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:22.966354 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:22.966323 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd"] Sep 10 22:35:22.970052 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:22.970025 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b902065f-r4lzd"] Sep 10 22:35:23.751079 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:23.751045 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35ac677-9efa-4999-8236-5b4b5b402a63" path="/var/lib/kubelet/pods/f35ac677-9efa-4999-8236-5b4b5b402a63/volumes" Sep 10 22:35:35.778969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:35.778927 2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 22:35:35.779375 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:35.778987 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 22:35:36.851070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:36.851033 2577 generic.go:356] "Generic (PLEG): container finished" podID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerID="b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4" exitCode=0 Sep 10 22:35:36.851494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:36.851110 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" event={"ID":"3eded38c-eb50-406d-8f02-8cdf3c4e83c0","Type":"ContainerDied","Data":"b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4"} Sep 10 22:35:36.851576 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:36.851560 2577 scope.go:117] "RemoveContainer" containerID="b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4" Sep 10 22:35:36.852584 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:36.852558 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:35:36.852638 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:36.852622 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:35:36.852680 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:36.852654 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:35:37.858765 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:37.858734 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" event={"ID":"3eded38c-eb50-406d-8f02-8cdf3c4e83c0","Type":"ContainerStarted","Data":"151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6"} Sep 10 22:35:38.231137 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:38.231041 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:38.231307 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:38.231282 2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 22:35:38.231360 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:38.231328 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 22:35:39.231046 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:39.231010 2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 22:35:39.231426 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:39.231071 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 22:35:40.231913 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:40.231874 2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 22:35:40.232314 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:40.231935 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 22:35:41.231374 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:41.231340 2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 22:35:41.231559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:41.231392 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 22:35:42.231253 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:42.231217 2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Sep 10 22:35:42.231649 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:42.231272 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Sep 10 22:35:43.234545 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:43.234513 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:43.235095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:43.234936 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:43.235640 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:43.235619 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:35:47.470205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.470166 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb"] Sep 10 22:35:47.474643 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.474621 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.483017 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.482997 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ljsfd\"" Sep 10 22:35:47.483209 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.483030 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 22:35:47.483296 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.483032 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-7a7e94-serving-cert\"" Sep 10 22:35:47.487359 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.487332 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb"] Sep 10 22:35:47.584104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.584068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e36069b-3d33-4cb5-9c91-be1cd37e032b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.584287 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.584255 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.584368 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.584316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.584424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.584402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdm7s\" (UniqueName: \"kubernetes.io/projected/8e36069b-3d33-4cb5-9c91-be1cd37e032b-kube-api-access-bdm7s\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.685245 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.685207 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e36069b-3d33-4cb5-9c91-be1cd37e032b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.685245 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.685250 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.685493 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.685282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.685493 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.685332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdm7s\" (UniqueName: \"kubernetes.io/projected/8e36069b-3d33-4cb5-9c91-be1cd37e032b-kube-api-access-bdm7s\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.685493 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:35:47.685459 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-serving-cert: secret "maas-api-e2e-shared-a-w2-7a7e94-serving-cert" not found Sep 10 22:35:47.685654 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:35:47.685522 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls podName:8e36069b-3d33-4cb5-9c91-be1cd37e032b nodeName:}" failed. No retries permitted until 2026-09-10 22:35:48.185502452 +0000 UTC m=+1054.981717910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls") pod "maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" (UID: "8e36069b-3d33-4cb5-9c91-be1cd37e032b") : secret "maas-api-e2e-shared-a-w2-7a7e94-serving-cert" not found Sep 10 22:35:47.686054 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.686028 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e36069b-3d33-4cb5-9c91-be1cd37e032b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.688080 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.688048 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:47.698095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:47.698072 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdm7s\" (UniqueName: \"kubernetes.io/projected/8e36069b-3d33-4cb5-9c91-be1cd37e032b-kube-api-access-bdm7s\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:48.190965 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:48.190933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:48.193958 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:48.193932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:48.389306 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:48.389254 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:48.662809 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:48.662785 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb"] Sep 10 22:35:48.668600 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:48.668570 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:35:48.934062 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:48.934029 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" event={"ID":"8e36069b-3d33-4cb5-9c91-be1cd37e032b","Type":"ContainerStarted","Data":"bbbb4f43e476b57355c28452ba6c886b3b52dc0923da30f20bd74d46fe41bd7b"} Sep 10 22:35:49.941125 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:49.941088 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" event={"ID":"8e36069b-3d33-4cb5-9c91-be1cd37e032b","Type":"ContainerStarted","Data":"3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680"} Sep 10 22:35:49.941509 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:49.941190 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:49.959376 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:49.959329 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" podStartSLOduration=2.619064969 podStartE2EDuration="2.959314225s" totalImagesPullingTime="340.249256ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:35:49.95777821 +0000 UTC m=+1056.753993688" watchObservedRunningTime="2026-09-10 22:35:49.959314225 +0000 UTC m=+1056.755529702" Sep 10 22:35:50.312728 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.312690 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m"] Sep 10 22:35:50.317317 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.317296 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.319358 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.319334 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-33b4cf-serving-cert\"" Sep 10 22:35:50.327981 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.327956 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m"] Sep 10 22:35:50.410767 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.410727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.411002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.410843 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.411002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.410898 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203be1d6-715b-4694-a338-f19ef1c53ea5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.411002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.410942 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj4zq\" (UniqueName: \"kubernetes.io/projected/203be1d6-715b-4694-a338-f19ef1c53ea5-kube-api-access-hj4zq\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.512046 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.512016 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.512214 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.512061 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203be1d6-715b-4694-a338-f19ef1c53ea5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.512214 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.512096 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj4zq\" (UniqueName: \"kubernetes.io/projected/203be1d6-715b-4694-a338-f19ef1c53ea5-kube-api-access-hj4zq\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.512214 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.512141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.512852 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.512757 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203be1d6-715b-4694-a338-f19ef1c53ea5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.514650 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.514627 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.514748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.514678 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.520396 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.520369 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj4zq\" (UniqueName: \"kubernetes.io/projected/203be1d6-715b-4694-a338-f19ef1c53ea5-kube-api-access-hj4zq\") pod \"maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.630037 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.629949 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:50.912506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.912480 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m"] Sep 10 22:35:50.914697 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:35:50.914669 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203be1d6_715b_4694_a338_f19ef1c53ea5.slice/crio-c37fc3e35e32cabceff7ccffb4d9c07316e904cd6ed28e1b312c136197b6f885 WatchSource:0}: Error finding container c37fc3e35e32cabceff7ccffb4d9c07316e904cd6ed28e1b312c136197b6f885: Status 404 returned error can't find the container with id c37fc3e35e32cabceff7ccffb4d9c07316e904cd6ed28e1b312c136197b6f885 Sep 10 22:35:50.948527 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:50.948491 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" event={"ID":"203be1d6-715b-4694-a338-f19ef1c53ea5","Type":"ContainerStarted","Data":"c37fc3e35e32cabceff7ccffb4d9c07316e904cd6ed28e1b312c136197b6f885"} Sep 10 22:35:51.955663 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:51.955627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" event={"ID":"203be1d6-715b-4694-a338-f19ef1c53ea5","Type":"ContainerStarted","Data":"7b8b89be83c3a82c09642547a11c27e6a4b37f017bcb9e2c47759c8d3efc872f"} Sep 10 22:35:51.956175 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:51.955754 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:35:51.974349 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:51.974298 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" podStartSLOduration=1.615283834 podStartE2EDuration="1.974284281s" totalImagesPullingTime="359.000447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:35:51.972544539 +0000 UTC m=+1058.768760017" watchObservedRunningTime="2026-09-10 22:35:51.974284281 +0000 UTC m=+1058.770499797" Sep 10 22:35:55.954686 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:55.954655 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:35:57.966915 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:35:57.966885 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:36:11.677277 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.677189 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr"] Sep 10 22:36:11.681073 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.681054 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.683311 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.683294 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-25e5bd-openshift-default-dockercfg-k2bth\"" Sep 10 22:36:11.696242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.696214 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr"] Sep 10 22:36:11.715245 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715213 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-token\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715252 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-socket\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715275 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-podinfo\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r845d\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-kube-api-access-r845d\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-credential-socket\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715411 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-envoy\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abfb18a5-4300-4dd3-afc7-e04309ac695f-istiod-ca-cert\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715926 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715521 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-certs\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.715926 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.715550 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-data\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.816762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.816726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-envoy\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.816987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.816771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abfb18a5-4300-4dd3-afc7-e04309ac695f-istiod-ca-cert\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.816987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.816811 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-certs\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.816987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.816860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-data\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.816987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.816901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-token\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.816987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.816934 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-socket\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.816987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.816959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-podinfo\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.817310 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.817278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r845d\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-kube-api-access-r845d\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.817369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.817337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-credential-socket\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.817428 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.817380 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-socket\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.817428 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.817336 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-certs\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.817532 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.817429 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-data\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.817665 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.817645 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-credential-socket\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.817751 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.817665 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abfb18a5-4300-4dd3-afc7-e04309ac695f-istiod-ca-cert\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.819455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.819430 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-envoy\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.819660 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.819643 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-podinfo\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.824615 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.824592 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-token\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.824733 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.824659 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r845d\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-kube-api-access-r845d\") pod \"e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:11.992663 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:11.992573 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:12.273200 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:12.273165 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr"] Sep 10 22:36:12.274939 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:36:12.274908 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabfb18a5_4300_4dd3_afc7_e04309ac695f.slice/crio-4a5be1d5423b5103dbeae862b05d25864e7dc6a2115d0ebe3d5c4e4573ca5372 WatchSource:0}: Error finding container 4a5be1d5423b5103dbeae862b05d25864e7dc6a2115d0ebe3d5c4e4573ca5372: Status 404 returned error can't find the container with id 4a5be1d5423b5103dbeae862b05d25864e7dc6a2115d0ebe3d5c4e4573ca5372 Sep 10 22:36:12.277448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:12.277416 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:12.277533 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:12.277484 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:12.277533 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:12.277515 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:13.082424 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:13.082390 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" event={"ID":"abfb18a5-4300-4dd3-afc7-e04309ac695f","Type":"ContainerStarted","Data":"5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b"} Sep 10 22:36:13.082922 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:13.082447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" event={"ID":"abfb18a5-4300-4dd3-afc7-e04309ac695f","Type":"ContainerStarted","Data":"4a5be1d5423b5103dbeae862b05d25864e7dc6a2115d0ebe3d5c4e4573ca5372"} Sep 10 22:36:13.108124 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:13.108077 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podStartSLOduration=2.108061625 podStartE2EDuration="2.108061625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:36:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:36:13.105218859 +0000 UTC m=+1079.901434357" watchObservedRunningTime="2026-09-10 22:36:13.108061625 +0000 UTC m=+1079.904277157" Sep 10 22:36:13.993789 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:13.993755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:13.999284 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:13.999260 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:14.089425 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:14.089395 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:14.090619 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:14.090594 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:34.215701 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:34.215661 2577 generic.go:356] "Generic (PLEG): container finished" podID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerID="5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b" exitCode=0 Sep 10 22:36:34.216161 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:34.215718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" event={"ID":"abfb18a5-4300-4dd3-afc7-e04309ac695f","Type":"ContainerDied","Data":"5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b"} Sep 10 22:36:34.216161 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:34.216119 2577 scope.go:117] "RemoveContainer" containerID="5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b" Sep 10 22:36:34.217206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:34.217174 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:34.217311 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:34.217259 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:34.217311 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:34.217305 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:35.223774 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:35.223740 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" event={"ID":"abfb18a5-4300-4dd3-afc7-e04309ac695f","Type":"ContainerStarted","Data":"8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784"} Sep 10 22:36:35.993437 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:35.993401 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:35.993721 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:35.993690 2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Sep 10 22:36:35.993798 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:35.993757 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Sep 10 22:36:36.993496 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:36.993462 2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Sep 10 22:36:36.994128 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:36.994097 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Sep 10 22:36:37.993941 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:37.993899 2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Sep 10 22:36:37.994435 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:37.993971 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Sep 10 22:36:38.993798 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:38.993753 2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Sep 10 22:36:38.994017 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:38.993844 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Sep 10 22:36:39.993415 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:39.993373 2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Sep 10 22:36:39.993627 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:39.993448 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Sep 10 22:36:40.783356 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.783327 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-46b10c"] Sep 10 22:36:40.786014 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.785994 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:40.788422 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.788403 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ks5gc\"" Sep 10 22:36:40.795605 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.795578 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-46b10c"] Sep 10 22:36:40.910950 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.910917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwkzp\" (UniqueName: \"kubernetes.io/projected/c897e1b7-3030-45bb-86cc-3e2cc5cf10d0-kube-api-access-nwkzp\") pod \"test-curl-7574-46b10c\" (UID: \"c897e1b7-3030-45bb-86cc-3e2cc5cf10d0\") " pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:40.997561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.997530 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:40.997935 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.997915 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:40.998505 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:40.998489 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:36:41.011907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:41.011874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwkzp\" (UniqueName: \"kubernetes.io/projected/c897e1b7-3030-45bb-86cc-3e2cc5cf10d0-kube-api-access-nwkzp\") pod \"test-curl-7574-46b10c\" (UID: \"c897e1b7-3030-45bb-86cc-3e2cc5cf10d0\") " pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:41.023266 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:41.023241 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwkzp\" (UniqueName: \"kubernetes.io/projected/c897e1b7-3030-45bb-86cc-3e2cc5cf10d0-kube-api-access-nwkzp\") pod \"test-curl-7574-46b10c\" (UID: \"c897e1b7-3030-45bb-86cc-3e2cc5cf10d0\") " pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:41.097329 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:41.097247 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:41.373184 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:41.373162 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-46b10c"] Sep 10 22:36:41.375368 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:36:41.375344 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc897e1b7_3030_45bb_86cc_3e2cc5cf10d0.slice/crio-6ea67e1e7680fe8e28fe36d149b03b3aa81fa3cd701f3ee7b4e654af3db8bfd0 WatchSource:0}: Error finding container 6ea67e1e7680fe8e28fe36d149b03b3aa81fa3cd701f3ee7b4e654af3db8bfd0: Status 404 returned error can't find the container with id 6ea67e1e7680fe8e28fe36d149b03b3aa81fa3cd701f3ee7b4e654af3db8bfd0 Sep 10 22:36:42.268987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:42.268950 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-46b10c" event={"ID":"c897e1b7-3030-45bb-86cc-3e2cc5cf10d0","Type":"ContainerStarted","Data":"6ea67e1e7680fe8e28fe36d149b03b3aa81fa3cd701f3ee7b4e654af3db8bfd0"} Sep 10 22:36:44.292700 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:44.292664 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-46b10c" event={"ID":"c897e1b7-3030-45bb-86cc-3e2cc5cf10d0","Type":"ContainerStarted","Data":"d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670"} Sep 10 22:36:44.309129 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:44.309085 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7574-46b10c" podStartSLOduration=2.259492801 podStartE2EDuration="4.30907128s" totalImagesPullingTime="2.049578479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:36:44.308361117 +0000 UTC m=+1111.104576595" watchObservedRunningTime="2026-09-10 22:36:44.30907128 +0000 UTC m=+1111.105286756" Sep 10 22:36:44.925171 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:44.925134 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-46b10c"] Sep 10 22:36:44.929606 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:44.929581 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-46b10c"] Sep 10 22:36:45.043026 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.042992 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-29bdda"] Sep 10 22:36:45.043457 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.043443 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" containerName="test-curl-7574-46b10c" Sep 10 22:36:45.043506 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.043459 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" containerName="test-curl-7574-46b10c" Sep 10 22:36:45.043577 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.043567 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" containerName="test-curl-7574-46b10c" Sep 10 22:36:45.045657 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.045641 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:45.052551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.052523 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-29bdda"] Sep 10 22:36:45.153917 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.153881 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2spn\" (UniqueName: \"kubernetes.io/projected/5998f0b6-ecf3-43d6-a957-662bf410dc6e-kube-api-access-h2spn\") pod \"test-curl-7574-29bdda\" (UID: \"5998f0b6-ecf3-43d6-a957-662bf410dc6e\") " pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:45.254597 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.254559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2spn\" (UniqueName: \"kubernetes.io/projected/5998f0b6-ecf3-43d6-a957-662bf410dc6e-kube-api-access-h2spn\") pod \"test-curl-7574-29bdda\" (UID: \"5998f0b6-ecf3-43d6-a957-662bf410dc6e\") " pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:45.262732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.262704 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2spn\" (UniqueName: \"kubernetes.io/projected/5998f0b6-ecf3-43d6-a957-662bf410dc6e-kube-api-access-h2spn\") pod \"test-curl-7574-29bdda\" (UID: \"5998f0b6-ecf3-43d6-a957-662bf410dc6e\") " pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:45.298662 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:45.298596 2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7574-46b10c" container="test-curl-7574-46b10c" Sep 10 22:36:45.355663 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.355632 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:45.628036 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:45.628013 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-29bdda"] Sep 10 22:36:45.629953 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:36:45.629924 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5998f0b6_ecf3_43d6_a957_662bf410dc6e.slice/crio-5ece18d6f17c95e673692c4cf9c471f1613d14e8b5553f68ef053bd897c3b926 WatchSource:0}: Error finding container 5ece18d6f17c95e673692c4cf9c471f1613d14e8b5553f68ef053bd897c3b926: Status 404 returned error can't find the container with id 5ece18d6f17c95e673692c4cf9c471f1613d14e8b5553f68ef053bd897c3b926 Sep 10 22:36:46.307234 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:46.307185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-29bdda" event={"ID":"5998f0b6-ecf3-43d6-a957-662bf410dc6e","Type":"ContainerStarted","Data":"868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff"} Sep 10 22:36:46.307234 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:46.307235 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-29bdda" event={"ID":"5998f0b6-ecf3-43d6-a957-662bf410dc6e","Type":"ContainerStarted","Data":"5ece18d6f17c95e673692c4cf9c471f1613d14e8b5553f68ef053bd897c3b926"} Sep 10 22:36:46.307727 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:46.307259 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7574-46b10c" podUID="c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" containerName="test-curl-7574-46b10c" containerID="cri-o://d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670" gracePeriod=2 Sep 10 22:36:46.917268 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:46.917215 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7574-29bdda" podStartSLOduration=1.917199801 podStartE2EDuration="1.917199801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:36:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:36:46.320690922 +0000 UTC m=+1113.116906400" watchObservedRunningTime="2026-09-10 22:36:46.917199801 +0000 UTC m=+1113.713415278" Sep 10 22:36:46.918096 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:46.918079 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-29bdda"] Sep 10 22:36:46.922616 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:46.922586 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-29bdda"] Sep 10 22:36:47.034325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.034294 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-431ae4"] Sep 10 22:36:47.034720 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.034707 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5998f0b6-ecf3-43d6-a957-662bf410dc6e" containerName="test-curl-7574-29bdda" Sep 10 22:36:47.034764 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.034721 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5998f0b6-ecf3-43d6-a957-662bf410dc6e" containerName="test-curl-7574-29bdda" Sep 10 22:36:47.035120 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.034807 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5998f0b6-ecf3-43d6-a957-662bf410dc6e" containerName="test-curl-7574-29bdda" Sep 10 22:36:47.037104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.037079 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:47.044073 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.044051 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-431ae4"] Sep 10 22:36:47.173248 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.173217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9dr9\" (UniqueName: \"kubernetes.io/projected/75e8194b-a214-4ca9-bd1d-14397e6f631c-kube-api-access-g9dr9\") pod \"test-curl-7574-431ae4\" (UID: \"75e8194b-a214-4ca9-bd1d-14397e6f631c\") " pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:47.274136 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.274105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9dr9\" (UniqueName: \"kubernetes.io/projected/75e8194b-a214-4ca9-bd1d-14397e6f631c-kube-api-access-g9dr9\") pod \"test-curl-7574-431ae4\" (UID: \"75e8194b-a214-4ca9-bd1d-14397e6f631c\") " pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:47.283191 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.283161 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9dr9\" (UniqueName: \"kubernetes.io/projected/75e8194b-a214-4ca9-bd1d-14397e6f631c-kube-api-access-g9dr9\") pod \"test-curl-7574-431ae4\" (UID: \"75e8194b-a214-4ca9-bd1d-14397e6f631c\") " pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:47.313477 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:47.313409 2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7574-29bdda" container="test-curl-7574-29bdda" Sep 10 22:36:47.347980 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.347949 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:47.623152 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:47.623121 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-431ae4"] Sep 10 22:36:47.624303 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:36:47.624277 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e8194b_a214_4ca9_bd1d_14397e6f631c.slice/crio-fcf1401a60b5bceee4b876b1d4f9bf2ac9bf9e4c62d56703009734b9aabadd8b WatchSource:0}: Error finding container fcf1401a60b5bceee4b876b1d4f9bf2ac9bf9e4c62d56703009734b9aabadd8b: Status 404 returned error can't find the container with id fcf1401a60b5bceee4b876b1d4f9bf2ac9bf9e4c62d56703009734b9aabadd8b Sep 10 22:36:48.321142 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.321060 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-431ae4" event={"ID":"75e8194b-a214-4ca9-bd1d-14397e6f631c","Type":"ContainerStarted","Data":"712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967"} Sep 10 22:36:48.321142 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.321109 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-431ae4" event={"ID":"75e8194b-a214-4ca9-bd1d-14397e6f631c","Type":"ContainerStarted","Data":"fcf1401a60b5bceee4b876b1d4f9bf2ac9bf9e4c62d56703009734b9aabadd8b"} Sep 10 22:36:48.321142 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.321110 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7574-29bdda" podUID="5998f0b6-ecf3-43d6-a957-662bf410dc6e" containerName="test-curl-7574-29bdda" containerID="cri-o://868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff" gracePeriod=2 Sep 10 22:36:48.333118 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.333045 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7574-431ae4" podStartSLOduration=1.333030607 podStartE2EDuration="1.333030607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:36:48.333022555 +0000 UTC m=+1115.129238033" watchObservedRunningTime="2026-09-10 22:36:48.333030607 +0000 UTC m=+1115.129246081" Sep 10 22:36:48.489531 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.489506 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:48.687382 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.687299 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwkzp\" (UniqueName: \"kubernetes.io/projected/c897e1b7-3030-45bb-86cc-3e2cc5cf10d0-kube-api-access-nwkzp\") pod \"c897e1b7-3030-45bb-86cc-3e2cc5cf10d0\" (UID: \"c897e1b7-3030-45bb-86cc-3e2cc5cf10d0\") " Sep 10 22:36:48.689588 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.689561 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c897e1b7-3030-45bb-86cc-3e2cc5cf10d0-kube-api-access-nwkzp" (OuterVolumeSpecName: "kube-api-access-nwkzp") pod "c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" (UID: "c897e1b7-3030-45bb-86cc-3e2cc5cf10d0"). InnerVolumeSpecName "kube-api-access-nwkzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:36:48.789550 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.789518 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwkzp\" (UniqueName: \"kubernetes.io/projected/c897e1b7-3030-45bb-86cc-3e2cc5cf10d0-kube-api-access-nwkzp\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:36:48.905442 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.905415 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-431ae4"] Sep 10 22:36:48.909544 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:48.909518 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-431ae4"] Sep 10 22:36:49.028800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.028770 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7574-1ac39e"] Sep 10 22:36:49.029220 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.029206 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e8194b-a214-4ca9-bd1d-14397e6f631c" containerName="test-curl-7574-431ae4" Sep 10 22:36:49.029263 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.029222 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e8194b-a214-4ca9-bd1d-14397e6f631c" containerName="test-curl-7574-431ae4" Sep 10 22:36:49.029321 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.029311 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e8194b-a214-4ca9-bd1d-14397e6f631c" containerName="test-curl-7574-431ae4" Sep 10 22:36:49.031490 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.031473 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:49.038522 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.038497 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-1ac39e"] Sep 10 22:36:49.091029 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.091004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jddmr\" (UniqueName: \"kubernetes.io/projected/1628843c-15f7-4c46-81ae-4b2417de167a-kube-api-access-jddmr\") pod \"test-curl-7574-1ac39e\" (UID: \"1628843c-15f7-4c46-81ae-4b2417de167a\") " pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:49.192364 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.192334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jddmr\" (UniqueName: \"kubernetes.io/projected/1628843c-15f7-4c46-81ae-4b2417de167a-kube-api-access-jddmr\") pod \"test-curl-7574-1ac39e\" (UID: \"1628843c-15f7-4c46-81ae-4b2417de167a\") " pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:49.200883 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.200858 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jddmr\" (UniqueName: \"kubernetes.io/projected/1628843c-15f7-4c46-81ae-4b2417de167a-kube-api-access-jddmr\") pod \"test-curl-7574-1ac39e\" (UID: \"1628843c-15f7-4c46-81ae-4b2417de167a\") " pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:49.327998 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.327915 2577 generic.go:356] "Generic (PLEG): container finished" podID="c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" containerID="d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670" exitCode=137 Sep 10 22:36:49.327998 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.327958 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:49.328418 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.328013 2577 scope.go:117] "RemoveContainer" containerID="d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670" Sep 10 22:36:49.328418 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:49.328287 2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7574-431ae4" container="test-curl-7574-431ae4" Sep 10 22:36:49.338599 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.338585 2577 scope.go:117] "RemoveContainer" containerID="d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670" Sep 10 22:36:49.338912 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:49.338891 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670\": container with ID starting with d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670 not found: ID does not exist" containerID="d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670" Sep 10 22:36:49.339002 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.338917 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670"} err="failed to get container status \"d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670\": rpc error: code = NotFound desc = could not find container \"d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670\": container with ID starting with d81c31a2a980464579dfd0e3c755189f550a117c594cee5cc3fde5eba46ca670 not found: ID does not exist" Sep 10 22:36:49.339050 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:49.339009 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-46b10c\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" pod="opendatahub/test-curl-7574-46b10c" Sep 10 22:36:49.343111 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.343095 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:49.617519 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.617412 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7574-1ac39e"] Sep 10 22:36:49.623219 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:36:49.623190 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1628843c_15f7_4c46_81ae_4b2417de167a.slice/crio-0ef43ce11ac96737bf4e45e9813224ad47f11433832183d4a7d7829185a9d728 WatchSource:0}: Error finding container 0ef43ce11ac96737bf4e45e9813224ad47f11433832183d4a7d7829185a9d728: Status 404 returned error can't find the container with id 0ef43ce11ac96737bf4e45e9813224ad47f11433832183d4a7d7829185a9d728 Sep 10 22:36:49.750269 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:49.750230 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c897e1b7-3030-45bb-86cc-3e2cc5cf10d0" path="/var/lib/kubelet/pods/c897e1b7-3030-45bb-86cc-3e2cc5cf10d0/volumes" Sep 10 22:36:50.336461 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.336426 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-1ac39e" event={"ID":"1628843c-15f7-4c46-81ae-4b2417de167a","Type":"ContainerStarted","Data":"dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f"} Sep 10 22:36:50.336862 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.336475 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7574-1ac39e" event={"ID":"1628843c-15f7-4c46-81ae-4b2417de167a","Type":"ContainerStarted","Data":"0ef43ce11ac96737bf4e45e9813224ad47f11433832183d4a7d7829185a9d728"} Sep 10 22:36:50.336862 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.336582 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7574-431ae4" podUID="75e8194b-a214-4ca9-bd1d-14397e6f631c" containerName="test-curl-7574-431ae4" containerID="cri-o://712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967" gracePeriod=2 Sep 10 22:36:50.350826 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.350772 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7574-1ac39e" podStartSLOduration=1.350756871 podStartE2EDuration="1.350756871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:36:50.349478064 +0000 UTC m=+1117.145693543" watchObservedRunningTime="2026-09-10 22:36:50.350756871 +0000 UTC m=+1117.146972348" Sep 10 22:36:50.495785 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.495766 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:50.504108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.504086 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2spn\" (UniqueName: \"kubernetes.io/projected/5998f0b6-ecf3-43d6-a957-662bf410dc6e-kube-api-access-h2spn\") pod \"5998f0b6-ecf3-43d6-a957-662bf410dc6e\" (UID: \"5998f0b6-ecf3-43d6-a957-662bf410dc6e\") " Sep 10 22:36:50.506218 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.506192 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5998f0b6-ecf3-43d6-a957-662bf410dc6e-kube-api-access-h2spn" (OuterVolumeSpecName: "kube-api-access-h2spn") pod "5998f0b6-ecf3-43d6-a957-662bf410dc6e" (UID: "5998f0b6-ecf3-43d6-a957-662bf410dc6e"). InnerVolumeSpecName "kube-api-access-h2spn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:36:50.604807 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.604736 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2spn\" (UniqueName: \"kubernetes.io/projected/5998f0b6-ecf3-43d6-a957-662bf410dc6e-kube-api-access-h2spn\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:36:50.926575 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.926547 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7574-1ac39e"] Sep 10 22:36:50.931302 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:50.931273 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7574-1ac39e"] Sep 10 22:36:51.343145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.343112 2577 generic.go:356] "Generic (PLEG): container finished" podID="5998f0b6-ecf3-43d6-a957-662bf410dc6e" containerID="868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff" exitCode=137 Sep 10 22:36:51.343145 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.343146 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:51.343593 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.343215 2577 scope.go:117] "RemoveContainer" containerID="868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff" Sep 10 22:36:51.343593 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:51.343424 2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7574-1ac39e" container="test-curl-7574-1ac39e" Sep 10 22:36:51.353159 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.353138 2577 scope.go:117] "RemoveContainer" containerID="868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff" Sep 10 22:36:51.353270 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:51.353207 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-29bdda\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="5998f0b6-ecf3-43d6-a957-662bf410dc6e" pod="opendatahub/test-curl-7574-29bdda" Sep 10 22:36:51.353404 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:51.353388 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff\": container with ID starting with 868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff not found: ID does not exist" containerID="868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff" Sep 10 22:36:51.353443 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.353412 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff"} err="failed to get container status \"868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff\": rpc error: code = NotFound desc = could not find container \"868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff\": container with ID starting with 868ff47eeead9aaabfe1a015572d96eb417090eb8194e1ed9ae63fbec57a2aff not found: ID does not exist" Sep 10 22:36:51.750270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.750196 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5998f0b6-ecf3-43d6-a957-662bf410dc6e" path="/var/lib/kubelet/pods/5998f0b6-ecf3-43d6-a957-662bf410dc6e/volumes" Sep 10 22:36:51.796276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.796235 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn"] Sep 10 22:36:51.796934 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.796916 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1628843c-15f7-4c46-81ae-4b2417de167a" containerName="test-curl-7574-1ac39e" Sep 10 22:36:51.797023 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.796936 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1628843c-15f7-4c46-81ae-4b2417de167a" containerName="test-curl-7574-1ac39e" Sep 10 22:36:51.797063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.797051 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1628843c-15f7-4c46-81ae-4b2417de167a" containerName="test-curl-7574-1ac39e" Sep 10 22:36:51.800277 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.800248 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.802503 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.802480 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-022f2a6a-openshift-default-dockercfg-k4x59\"" Sep 10 22:36:51.811601 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.811576 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn"] Sep 10 22:36:51.814766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.814741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-istio-token\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.814920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.814771 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-envoy\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.814920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.814789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwzzb\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-kube-api-access-vwzzb\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.814920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.814847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c756932-5e01-46da-be7d-e730cc7c001b-istiod-ca-cert\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.815103 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.814972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-certs\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.815103 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.815081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-credential-socket\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.815205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.815121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-data\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.815205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.815152 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-socket\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.815205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.815173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c756932-5e01-46da-be7d-e730cc7c001b-istio-podinfo\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916010 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.915968 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-credential-socket\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916010 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-data\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916047 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-socket\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916068 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c756932-5e01-46da-be7d-e730cc7c001b-istio-podinfo\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916123 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-istio-token\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-envoy\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwzzb\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-kube-api-access-vwzzb\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916237 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c756932-5e01-46da-be7d-e730cc7c001b-istiod-ca-cert\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916369 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916304 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-certs\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916418 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-credential-socket\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916487 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-socket\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916496 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-data\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.916737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.916652 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-certs\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.917387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.917309 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c756932-5e01-46da-be7d-e730cc7c001b-istiod-ca-cert\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.919976 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.919929 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-envoy\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.921652 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.921518 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c756932-5e01-46da-be7d-e730cc7c001b-istio-podinfo\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.925059 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.925024 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-istio-token\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:51.925195 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:51.925153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwzzb\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-kube-api-access-vwzzb\") pod \"e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:52.113796 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.113765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:52.350222 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.350189 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7574-1ac39e" podUID="1628843c-15f7-4c46-81ae-4b2417de167a" containerName="test-curl-7574-1ac39e" containerID="cri-o://dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f" gracePeriod=2 Sep 10 22:36:52.397753 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.397725 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn"] Sep 10 22:36:52.406670 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:36:52.406637 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c756932_5e01_46da_be7d_e730cc7c001b.slice/crio-e6e2597415f8d45842bbf892306292ab518c5887b88cea307273da76c47a4c83 WatchSource:0}: Error finding container e6e2597415f8d45842bbf892306292ab518c5887b88cea307273da76c47a4c83: Status 404 returned error can't find the container with id e6e2597415f8d45842bbf892306292ab518c5887b88cea307273da76c47a4c83 Sep 10 22:36:52.409161 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.409120 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:52.409289 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.409203 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:52.409289 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.409250 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:36:52.515701 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.515568 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:52.522189 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.522160 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9dr9\" (UniqueName: \"kubernetes.io/projected/75e8194b-a214-4ca9-bd1d-14397e6f631c-kube-api-access-g9dr9\") pod \"75e8194b-a214-4ca9-bd1d-14397e6f631c\" (UID: \"75e8194b-a214-4ca9-bd1d-14397e6f631c\") " Sep 10 22:36:52.524597 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.524566 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e8194b-a214-4ca9-bd1d-14397e6f631c-kube-api-access-g9dr9" (OuterVolumeSpecName: "kube-api-access-g9dr9") pod "75e8194b-a214-4ca9-bd1d-14397e6f631c" (UID: "75e8194b-a214-4ca9-bd1d-14397e6f631c"). InnerVolumeSpecName "kube-api-access-g9dr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:36:52.624070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:52.623922 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9dr9\" (UniqueName: \"kubernetes.io/projected/75e8194b-a214-4ca9-bd1d-14397e6f631c-kube-api-access-g9dr9\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:36:53.357938 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.357905 2577 generic.go:356] "Generic (PLEG): container finished" podID="75e8194b-a214-4ca9-bd1d-14397e6f631c" containerID="712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967" exitCode=137 Sep 10 22:36:53.358412 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.357952 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:53.358412 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.358006 2577 scope.go:117] "RemoveContainer" containerID="712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967" Sep 10 22:36:53.359974 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.359949 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" event={"ID":"5c756932-5e01-46da-be7d-e730cc7c001b","Type":"ContainerStarted","Data":"c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c"} Sep 10 22:36:53.360109 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.359995 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" event={"ID":"5c756932-5e01-46da-be7d-e730cc7c001b","Type":"ContainerStarted","Data":"e6e2597415f8d45842bbf892306292ab518c5887b88cea307273da76c47a4c83"} Sep 10 22:36:53.368189 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.368168 2577 scope.go:117] "RemoveContainer" containerID="712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967" Sep 10 22:36:53.368470 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:53.368451 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967\": container with ID starting with 712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967 not found: ID does not exist" containerID="712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967" Sep 10 22:36:53.368515 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.368478 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967"} err="failed to get container status \"712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967\": rpc error: code = NotFound desc = could not find container \"712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967\": container with ID starting with 712874ee1f69eab5f15041c8dd9bfc5017c14e45051702de20bebd370ac55967 not found: ID does not exist" Sep 10 22:36:53.379737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.379667 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" podStartSLOduration=2.379655294 podStartE2EDuration="2.379655294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:36:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:36:53.378187107 +0000 UTC m=+1120.174402585" watchObservedRunningTime="2026-09-10 22:36:53.379655294 +0000 UTC m=+1120.175870771" Sep 10 22:36:53.379888 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:53.379868 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-431ae4\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="75e8194b-a214-4ca9-bd1d-14397e6f631c" pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:53.749985 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:53.749904 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-431ae4\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="75e8194b-a214-4ca9-bd1d-14397e6f631c" pod="opendatahub/test-curl-7574-431ae4" Sep 10 22:36:53.751527 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:53.751505 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e8194b-a214-4ca9-bd1d-14397e6f631c" path="/var/lib/kubelet/pods/75e8194b-a214-4ca9-bd1d-14397e6f631c/volumes" Sep 10 22:36:54.114988 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.114955 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:54.120386 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.120361 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:54.368191 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.368158 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:54.369174 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.369155 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:36:54.535133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.535110 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:54.544063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.544041 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jddmr\" (UniqueName: \"kubernetes.io/projected/1628843c-15f7-4c46-81ae-4b2417de167a-kube-api-access-jddmr\") pod \"1628843c-15f7-4c46-81ae-4b2417de167a\" (UID: \"1628843c-15f7-4c46-81ae-4b2417de167a\") " Sep 10 22:36:54.546056 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.546033 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1628843c-15f7-4c46-81ae-4b2417de167a-kube-api-access-jddmr" (OuterVolumeSpecName: "kube-api-access-jddmr") pod "1628843c-15f7-4c46-81ae-4b2417de167a" (UID: "1628843c-15f7-4c46-81ae-4b2417de167a"). InnerVolumeSpecName "kube-api-access-jddmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:36:54.644850 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:54.644756 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jddmr\" (UniqueName: \"kubernetes.io/projected/1628843c-15f7-4c46-81ae-4b2417de167a-kube-api-access-jddmr\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:36:55.375914 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:55.375874 2577 generic.go:356] "Generic (PLEG): container finished" podID="1628843c-15f7-4c46-81ae-4b2417de167a" containerID="dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f" exitCode=137 Sep 10 22:36:55.375914 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:55.375910 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:55.376378 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:55.375971 2577 scope.go:117] "RemoveContainer" containerID="dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f" Sep 10 22:36:55.385448 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:55.385416 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7574-1ac39e\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="1628843c-15f7-4c46-81ae-4b2417de167a" pod="opendatahub/test-curl-7574-1ac39e" Sep 10 22:36:55.386047 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:55.386030 2577 scope.go:117] "RemoveContainer" containerID="dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f" Sep 10 22:36:55.386384 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:36:55.386356 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f\": container with ID starting with dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f not found: ID does not exist" containerID="dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f" Sep 10 22:36:55.386487 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:55.386394 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f"} err="failed to get container status \"dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f\": rpc error: code = NotFound desc = could not find container \"dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f\": container with ID starting with dd8d649479294f354b347979d88d223271e26e6137e1cd3f7ffce41062082c1f not found: ID does not exist" Sep 10 22:36:55.750265 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:36:55.750185 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1628843c-15f7-4c46-81ae-4b2417de167a" path="/var/lib/kubelet/pods/1628843c-15f7-4c46-81ae-4b2417de167a/volumes" Sep 10 22:37:18.529165 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:18.529129 2577 generic.go:356] "Generic (PLEG): container finished" podID="5c756932-5e01-46da-be7d-e730cc7c001b" containerID="c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c" exitCode=0 Sep 10 22:37:18.529747 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:18.529202 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" event={"ID":"5c756932-5e01-46da-be7d-e730cc7c001b","Type":"ContainerDied","Data":"c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c"} Sep 10 22:37:18.529747 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:18.529585 2577 scope.go:117] "RemoveContainer" containerID="c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c" Sep 10 22:37:18.530719 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:18.530688 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:37:18.530804 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:18.530757 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:37:18.530804 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:18.530793 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:37:19.538057 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:19.538023 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" event={"ID":"5c756932-5e01-46da-be7d-e730cc7c001b","Type":"ContainerStarted","Data":"00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2"} Sep 10 22:37:20.114510 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:20.114476 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:37:20.115621 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:20.115587 2577 patch_prober.go:28] interesting pod/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 22:37:20.115782 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:20.115652 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 22:37:21.114338 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:21.114286 2577 patch_prober.go:28] interesting pod/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 22:37:21.114872 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:21.114358 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 22:37:22.114289 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:22.114243 2577 patch_prober.go:28] interesting pod/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 22:37:22.114494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:22.114305 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 22:37:23.114955 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:23.114912 2577 patch_prober.go:28] interesting pod/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 22:37:23.115460 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:23.114983 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 22:37:24.115237 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:24.115192 2577 patch_prober.go:28] interesting pod/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Sep 10 22:37:24.115741 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:24.115261 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Sep 10 22:37:24.369100 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:24.369017 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:37:25.118267 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:25.118233 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:37:25.119359 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:37:25.119338 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:38:13.750667 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:38:13.750639 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:38:13.755244 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:38:13.755223 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:39:03.583478 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.583443 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7577-f1faaa"] Sep 10 22:39:03.586516 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.586491 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:03.588874 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.588853 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ks5gc\"" Sep 10 22:39:03.594016 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.593937 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7577-f1faaa"] Sep 10 22:39:03.703850 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.703796 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5grk\" (UniqueName: \"kubernetes.io/projected/c299fda8-3d2a-427d-9ab8-ce1f58f06a5e-kube-api-access-j5grk\") pod \"test-curl-7577-f1faaa\" (UID: \"c299fda8-3d2a-427d-9ab8-ce1f58f06a5e\") " pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:03.804694 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.804661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5grk\" (UniqueName: \"kubernetes.io/projected/c299fda8-3d2a-427d-9ab8-ce1f58f06a5e-kube-api-access-j5grk\") pod \"test-curl-7577-f1faaa\" (UID: \"c299fda8-3d2a-427d-9ab8-ce1f58f06a5e\") " pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:03.812695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.812666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5grk\" (UniqueName: \"kubernetes.io/projected/c299fda8-3d2a-427d-9ab8-ce1f58f06a5e-kube-api-access-j5grk\") pod \"test-curl-7577-f1faaa\" (UID: \"c299fda8-3d2a-427d-9ab8-ce1f58f06a5e\") " pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:03.898642 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:03.898551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:04.174086 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:04.174052 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7577-f1faaa"] Sep 10 22:39:04.206085 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:04.206058 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7577-f1faaa" event={"ID":"c299fda8-3d2a-427d-9ab8-ce1f58f06a5e","Type":"ContainerStarted","Data":"eb221d6372aa4fe59abde20e88ff683b1e0a93fb81f95bef021d89cd5584ef65"} Sep 10 22:39:05.213748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:05.213709 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7577-f1faaa" event={"ID":"c299fda8-3d2a-427d-9ab8-ce1f58f06a5e","Type":"ContainerStarted","Data":"2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c"} Sep 10 22:39:05.229637 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:05.229590 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7577-f1faaa" podStartSLOduration=2.22957525 podStartE2EDuration="2.22957525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:39:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:39:05.229330281 +0000 UTC m=+1252.025545759" watchObservedRunningTime="2026-09-10 22:39:05.22957525 +0000 UTC m=+1252.025790728" Sep 10 22:39:05.820831 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:05.820786 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7577-f1faaa"] Sep 10 22:39:05.824762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:05.824719 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7577-f1faaa"] Sep 10 22:39:06.219562 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:39:06.219523 2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7577-f1faaa" container="test-curl-7577-f1faaa" Sep 10 22:39:07.226688 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:07.226630 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7577-f1faaa" podUID="c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" containerName="test-curl-7577-f1faaa" containerID="cri-o://2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c" gracePeriod=2 Sep 10 22:39:07.230484 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:39:07.229067 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7577-f1faaa\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:09.419666 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:09.419635 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:09.558158 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:09.558127 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5grk\" (UniqueName: \"kubernetes.io/projected/c299fda8-3d2a-427d-9ab8-ce1f58f06a5e-kube-api-access-j5grk\") pod \"c299fda8-3d2a-427d-9ab8-ce1f58f06a5e\" (UID: \"c299fda8-3d2a-427d-9ab8-ce1f58f06a5e\") " Sep 10 22:39:09.560274 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:09.560244 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c299fda8-3d2a-427d-9ab8-ce1f58f06a5e-kube-api-access-j5grk" (OuterVolumeSpecName: "kube-api-access-j5grk") pod "c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" (UID: "c299fda8-3d2a-427d-9ab8-ce1f58f06a5e"). InnerVolumeSpecName "kube-api-access-j5grk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:39:09.658995 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:09.658965 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5grk\" (UniqueName: \"kubernetes.io/projected/c299fda8-3d2a-427d-9ab8-ce1f58f06a5e-kube-api-access-j5grk\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:09.756180 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:09.756146 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" path="/var/lib/kubelet/pods/c299fda8-3d2a-427d-9ab8-ce1f58f06a5e/volumes" Sep 10 22:39:10.250893 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:10.250860 2577 generic.go:356] "Generic (PLEG): container finished" podID="c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" containerID="2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c" exitCode=137 Sep 10 22:39:10.251083 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:10.250898 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-f1faaa" Sep 10 22:39:10.251083 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:10.250909 2577 scope.go:117] "RemoveContainer" containerID="2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c" Sep 10 22:39:10.260939 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:10.260921 2577 scope.go:117] "RemoveContainer" containerID="2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c" Sep 10 22:39:10.261225 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:39:10.261207 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c\": container with ID starting with 2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c not found: ID does not exist" containerID="2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c" Sep 10 22:39:10.261276 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:10.261233 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c"} err="failed to get container status \"2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c\": rpc error: code = NotFound desc = could not find container \"2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c\": container with ID starting with 2ffbaed89fc8251b52ec1ecd1c353e2fa70cf205d824fe58cb2dda2f0174a73c not found: ID does not exist" Sep 10 22:39:11.540459 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:11.540422 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn"] Sep 10 22:39:11.540887 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:11.540625 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" containerID="cri-o://00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2" gracePeriod=30 Sep 10 22:39:16.810236 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.810171 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:39:16.928997 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.928969 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwzzb\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-kube-api-access-vwzzb\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929153 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929018 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-envoy\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929153 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929064 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-certs\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929153 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929116 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-credential-socket\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929153 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929145 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-istio-token\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929167 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-data\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929197 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-socket\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929293 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c756932-5e01-46da-be7d-e730cc7c001b-istiod-ca-cert\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929326 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c756932-5e01-46da-be7d-e730cc7c001b-istio-podinfo\") pod \"5c756932-5e01-46da-be7d-e730cc7c001b\" (UID: \"5c756932-5e01-46da-be7d-e730cc7c001b\") " Sep 10 22:39:16.929571 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929387 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:39:16.929571 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929456 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:39:16.929571 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929511 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:39:16.929733 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929670 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:39:16.929733 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929691 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c756932-5e01-46da-be7d-e730cc7c001b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:39:16.930015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929911 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:16.930015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929934 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:16.930015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929948 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c756932-5e01-46da-be7d-e730cc7c001b-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:16.930015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929962 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:16.930015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.929975 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:16.931497 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.931469 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:39:16.931647 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.931525 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-kube-api-access-vwzzb" (OuterVolumeSpecName: "kube-api-access-vwzzb") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "kube-api-access-vwzzb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:39:16.931647 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.931625 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:39:16.931761 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:16.931746 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c756932-5e01-46da-be7d-e730cc7c001b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c756932-5e01-46da-be7d-e730cc7c001b" (UID: "5c756932-5e01-46da-be7d-e730cc7c001b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:39:17.030523 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.030494 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:17.030523 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.030517 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c756932-5e01-46da-be7d-e730cc7c001b-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:17.030523 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.030525 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwzzb\" (UniqueName: \"kubernetes.io/projected/5c756932-5e01-46da-be7d-e730cc7c001b-kube-api-access-vwzzb\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:17.030726 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.030536 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c756932-5e01-46da-be7d-e730cc7c001b-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:39:17.297396 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.297345 2577 generic.go:356] "Generic (PLEG): container finished" podID="5c756932-5e01-46da-be7d-e730cc7c001b" containerID="00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2" exitCode=0 Sep 10 22:39:17.297592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.297410 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" Sep 10 22:39:17.297592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.297423 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" event={"ID":"5c756932-5e01-46da-be7d-e730cc7c001b","Type":"ContainerDied","Data":"00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2"} Sep 10 22:39:17.297592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.297491 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn" event={"ID":"5c756932-5e01-46da-be7d-e730cc7c001b","Type":"ContainerDied","Data":"e6e2597415f8d45842bbf892306292ab518c5887b88cea307273da76c47a4c83"} Sep 10 22:39:17.297592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.297521 2577 scope.go:117] "RemoveContainer" containerID="00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2" Sep 10 22:39:17.308409 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.308392 2577 scope.go:117] "RemoveContainer" containerID="c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c" Sep 10 22:39:17.318775 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.318750 2577 scope.go:117] "RemoveContainer" containerID="00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2" Sep 10 22:39:17.319210 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:39:17.319186 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2\": container with ID starting with 00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2 not found: ID does not exist" containerID="00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2" Sep 10 22:39:17.319308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.319223 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2"} err="failed to get container status \"00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2\": rpc error: code = NotFound desc = could not find container \"00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2\": container with ID starting with 00991b3a4225a8e18bee9d63737bbd4af2d15e4e7d44ff46fa347d5e2e97dec2 not found: ID does not exist" Sep 10 22:39:17.319308 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.319242 2577 scope.go:117] "RemoveContainer" containerID="c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c" Sep 10 22:39:17.319545 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:39:17.319525 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c\": container with ID starting with c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c not found: ID does not exist" containerID="c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c" Sep 10 22:39:17.319587 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.319553 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c"} err="failed to get container status \"c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c\": rpc error: code = NotFound desc = could not find container \"c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c\": container with ID starting with c0dab42677a28cf5046305cd60a3bcde1ecca23ce966148541f8d8260603811c not found: ID does not exist" Sep 10 22:39:17.321448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.321429 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn"] Sep 10 22:39:17.325310 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.325289 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-022f2a6a-openshift-default-6576f78b4-64gkn"] Sep 10 22:39:17.750122 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:39:17.750044 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" path="/var/lib/kubelet/pods/5c756932-5e01-46da-be7d-e730cc7c001b/volumes" Sep 10 22:40:05.629306 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.629263 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p"] Sep 10 22:40:05.629959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.629939 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" Sep 10 22:40:05.629959 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.629961 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" Sep 10 22:40:05.630108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.629974 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" Sep 10 22:40:05.630108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.629982 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" Sep 10 22:40:05.630108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.630003 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" containerName="test-curl-7577-f1faaa" Sep 10 22:40:05.630108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.630013 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" containerName="test-curl-7577-f1faaa" Sep 10 22:40:05.630309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.630115 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c299fda8-3d2a-427d-9ab8-ce1f58f06a5e" containerName="test-curl-7577-f1faaa" Sep 10 22:40:05.630309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.630129 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" Sep 10 22:40:05.630309 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.630139 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c756932-5e01-46da-be7d-e730cc7c001b" containerName="istio-proxy" Sep 10 22:40:05.633499 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.633480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.636187 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.636167 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-03e78e-openshift-default-dockercfg-7wl75\"" Sep 10 22:40:05.653869 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.653834 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p"] Sep 10 22:40:05.768642 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-credential-socket\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.768642 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-envoy\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.768921 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768661 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-token\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.768921 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-socket\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.768921 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768717 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77npw\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-kube-api-access-77npw\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.768921 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-podinfo\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.768921 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768795 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-certs\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.768921 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-data\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.769147 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.768923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d8fd23e-55a0-444d-973c-5daee5fd4364-istiod-ca-cert\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.869807 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.869768 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-socket\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.869992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.869809 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77npw\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-kube-api-access-77npw\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.869992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.869876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-podinfo\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-certs\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-data\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870163 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d8fd23e-55a0-444d-973c-5daee5fd4364-istiod-ca-cert\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870242 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-socket\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870294 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870280 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-credential-socket\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870347 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870333 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-envoy\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870386 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870367 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-token\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870422 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870388 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-certs\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870500 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870480 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-data\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.870750 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870732 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-credential-socket\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.871015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.870996 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d8fd23e-55a0-444d-973c-5daee5fd4364-istiod-ca-cert\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.872778 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.872756 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-envoy\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.873022 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.873006 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-podinfo\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.878125 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.878102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-token\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.878279 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.878262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77npw\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-kube-api-access-77npw\") pod \"e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:05.946427 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:05.946338 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:06.228388 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.228351 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p"] Sep 10 22:40:06.233669 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.233636 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:06.233762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.233710 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:06.233762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.233738 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:06.599598 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.599555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" event={"ID":"9d8fd23e-55a0-444d-973c-5daee5fd4364","Type":"ContainerStarted","Data":"f58c99ff56800ec3b84621e52251ba16fac5822ea50951b7d29db378fecda0d1"} Sep 10 22:40:06.599850 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.599627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" event={"ID":"9d8fd23e-55a0-444d-973c-5daee5fd4364","Type":"ContainerStarted","Data":"8dad3e42ab2c7a2e4809c444475e388ad7a02a8c18d9487b0eaac00bd912dcb7"} Sep 10 22:40:06.620988 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.620938 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" podStartSLOduration=1.6209221 podStartE2EDuration="1.6209221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:40:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:40:06.619086674 +0000 UTC m=+1313.415302152" watchObservedRunningTime="2026-09-10 22:40:06.6209221 +0000 UTC m=+1313.417137603" Sep 10 22:40:06.946754 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:06.946678 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:07.952831 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:07.952777 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:08.612763 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:08.612734 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:08.613692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:08.613675 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:28.748217 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:28.748116 2577 generic.go:356] "Generic (PLEG): container finished" podID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerID="f58c99ff56800ec3b84621e52251ba16fac5822ea50951b7d29db378fecda0d1" exitCode=0 Sep 10 22:40:28.748217 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:28.748141 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" event={"ID":"9d8fd23e-55a0-444d-973c-5daee5fd4364","Type":"ContainerDied","Data":"f58c99ff56800ec3b84621e52251ba16fac5822ea50951b7d29db378fecda0d1"} Sep 10 22:40:28.748651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:28.748494 2577 scope.go:117] "RemoveContainer" containerID="f58c99ff56800ec3b84621e52251ba16fac5822ea50951b7d29db378fecda0d1" Sep 10 22:40:28.749697 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:28.749673 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:28.749764 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:28.749735 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:28.749803 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:28.749767 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:29.756292 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:29.756254 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" event={"ID":"9d8fd23e-55a0-444d-973c-5daee5fd4364","Type":"ContainerStarted","Data":"fd17bff6eab4dcfb8acfb5c6fbed748e5cbdbcf6adf653d91ef8e4487d26543b"} Sep 10 22:40:29.946516 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:29.946487 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:29.946748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:29.946720 2577 patch_prober.go:28] interesting pod/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Sep 10 22:40:29.946864 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:29.946785 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Sep 10 22:40:30.947143 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:30.947108 2577 patch_prober.go:28] interesting pod/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Sep 10 22:40:30.947530 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:30.947175 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Sep 10 22:40:31.947461 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:31.947414 2577 patch_prober.go:28] interesting pod/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Sep 10 22:40:31.947982 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:31.947484 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Sep 10 22:40:32.947103 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:32.947058 2577 patch_prober.go:28] interesting pod/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Sep 10 22:40:32.947319 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:32.947127 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Sep 10 22:40:33.947331 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:33.947300 2577 patch_prober.go:28] interesting pod/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Sep 10 22:40:33.947742 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:33.947352 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Sep 10 22:40:34.422591 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.422552 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r"] Sep 10 22:40:34.425583 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.425557 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.428735 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.428715 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ecb38e-openshift-default-dockercfg-6pvdk\"" Sep 10 22:40:34.451323 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.451265 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r"] Sep 10 22:40:34.557775 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.557727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-data\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558013 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.557798 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-envoy\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558013 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.557885 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-credential-socket\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558013 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.557907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-podinfo\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558013 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.557967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-token\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.558033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-certs\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.558058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpfsb\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-kube-api-access-kpfsb\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.558118 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istiod-ca-cert\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.558199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.558157 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-socket\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.658904 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.658861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-data\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659097 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.658926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-envoy\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659097 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.658986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-credential-socket\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659097 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-podinfo\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659097 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-token\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-certs\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659145 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpfsb\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-kube-api-access-kpfsb\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istiod-ca-cert\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659244 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-socket\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659327 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-data\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659598 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659497 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-certs\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659680 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659601 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-socket\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659962 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istiod-ca-cert\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.659985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.659975 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-credential-socket\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.661372 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.661348 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-envoy\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.661542 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.661525 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-podinfo\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.668262 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.668240 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-token\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.672342 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.672317 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpfsb\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-kube-api-access-kpfsb\") pod \"e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.736846 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.736742 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:34.950565 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.950535 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:34.951069 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.950929 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:34.951719 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:34.951699 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:40:35.045783 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:35.045752 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r"] Sep 10 22:40:35.047630 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:40:35.047581 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd3d0d05_a2af_424a_a8f5_b3d62721f1ce.slice/crio-f73716b7a72a82068fb32ad81599b1782a879f9fdc1cb1b01339696db956a71c WatchSource:0}: Error finding container f73716b7a72a82068fb32ad81599b1782a879f9fdc1cb1b01339696db956a71c: Status 404 returned error can't find the container with id f73716b7a72a82068fb32ad81599b1782a879f9fdc1cb1b01339696db956a71c Sep 10 22:40:35.050281 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:35.050245 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:35.050396 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:35.050323 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:35.050396 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:35.050364 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:35.796887 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:35.796850 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" event={"ID":"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce","Type":"ContainerStarted","Data":"52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b"} Sep 10 22:40:35.797050 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:35.796903 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" event={"ID":"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce","Type":"ContainerStarted","Data":"f73716b7a72a82068fb32ad81599b1782a879f9fdc1cb1b01339696db956a71c"} Sep 10 22:40:35.830107 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:35.830061 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podStartSLOduration=1.8300454739999998 podStartE2EDuration="1.830045474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:40:35.828646411 +0000 UTC m=+1342.624861891" watchObservedRunningTime="2026-09-10 22:40:35.830045474 +0000 UTC m=+1342.626260951" Sep 10 22:40:36.737664 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:36.737631 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:36.743156 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:36.743132 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:36.803245 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:36.803204 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:36.803872 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:36.803849 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:39.086159 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.086128 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9"] Sep 10 22:40:39.089239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.089216 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.091987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.091964 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ecb38e-serving-cert\"" Sep 10 22:40:39.106537 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.106513 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9"] Sep 10 22:40:39.202443 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.202394 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-metrics-certs\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.202799 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.202778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zpj\" (UniqueName: \"kubernetes.io/projected/82b544c3-a4c1-447f-b292-c387f8a0b40a-kube-api-access-z6zpj\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.203030 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.203011 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82b544c3-a4c1-447f-b292-c387f8a0b40a-openshift-service-ca\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.203178 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.203160 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.304629 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.304599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-metrics-certs\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.304766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.304662 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zpj\" (UniqueName: \"kubernetes.io/projected/82b544c3-a4c1-447f-b292-c387f8a0b40a-kube-api-access-z6zpj\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.304766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.304728 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82b544c3-a4c1-447f-b292-c387f8a0b40a-openshift-service-ca\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.304888 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.304763 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.304967 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:40:39.304950 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-serving-cert: secret "maas-api-e2e-api-b-ecb38e-serving-cert" not found Sep 10 22:40:39.305036 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:40:39.305025 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls podName:82b544c3-a4c1-447f-b292-c387f8a0b40a nodeName:}" failed. No retries permitted until 2026-09-10 22:40:39.805004115 +0000 UTC m=+1346.601219589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls") pod "maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" (UID: "82b544c3-a4c1-447f-b292-c387f8a0b40a") : secret "maas-api-e2e-api-b-ecb38e-serving-cert" not found Sep 10 22:40:39.305404 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.305377 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82b544c3-a4c1-447f-b292-c387f8a0b40a-openshift-service-ca\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.307230 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.307205 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-metrics-certs\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.314134 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.314110 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zpj\" (UniqueName: \"kubernetes.io/projected/82b544c3-a4c1-447f-b292-c387f8a0b40a-kube-api-access-z6zpj\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.810020 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.809986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:39.812658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:39.812629 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls\") pod \"maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:40.001393 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:40.001356 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:40.282481 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:40.282447 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9"] Sep 10 22:40:40.283690 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:40:40.283650 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b544c3_a4c1_447f_b292_c387f8a0b40a.slice/crio-056fbea85c383c29736c6230850063da7ba15c0ef41fac8452e61d8133400bc5 WatchSource:0}: Error finding container 056fbea85c383c29736c6230850063da7ba15c0ef41fac8452e61d8133400bc5: Status 404 returned error can't find the container with id 056fbea85c383c29736c6230850063da7ba15c0ef41fac8452e61d8133400bc5 Sep 10 22:40:40.834795 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:40.834703 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" event={"ID":"82b544c3-a4c1-447f-b292-c387f8a0b40a","Type":"ContainerStarted","Data":"66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32"} Sep 10 22:40:40.834795 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:40.834754 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" event={"ID":"82b544c3-a4c1-447f-b292-c387f8a0b40a","Type":"ContainerStarted","Data":"056fbea85c383c29736c6230850063da7ba15c0ef41fac8452e61d8133400bc5"} Sep 10 22:40:40.834795 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:40.834776 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:40.875448 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:40.875402 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" podStartSLOduration=1.586515221 podStartE2EDuration="1.87538664s" totalImagesPullingTime="288.871419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:40:40.873404221 +0000 UTC m=+1347.669619699" watchObservedRunningTime="2026-09-10 22:40:40.87538664 +0000 UTC m=+1347.671602116" Sep 10 22:40:46.846000 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:46.845971 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:40:52.843975 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:52.843926 2577 patch_prober.go:28] interesting pod/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 22:40:52.844371 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:52.843991 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 22:40:53.919525 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:53.919490 2577 generic.go:356] "Generic (PLEG): container finished" podID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerID="52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b" exitCode=0 Sep 10 22:40:53.919985 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:53.919563 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" event={"ID":"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce","Type":"ContainerDied","Data":"52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b"} Sep 10 22:40:53.920041 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:53.919997 2577 scope.go:117] "RemoveContainer" containerID="52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b" Sep 10 22:40:53.921185 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:53.921160 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:53.921239 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:53.921225 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:53.921277 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:53.921261 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:40:54.928072 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:54.928036 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" event={"ID":"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce","Type":"ContainerStarted","Data":"b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31"} Sep 10 22:40:55.737412 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:55.737380 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:40:55.737629 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:55.737577 2577 patch_prober.go:28] interesting pod/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Sep 10 22:40:55.737679 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:55.737633 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Sep 10 22:40:56.737636 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:56.737598 2577 patch_prober.go:28] interesting pod/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Sep 10 22:40:56.738096 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:56.737651 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Sep 10 22:40:57.737421 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:57.737382 2577 patch_prober.go:28] interesting pod/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Sep 10 22:40:57.737604 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:57.737439 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Sep 10 22:40:58.737875 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:58.737838 2577 patch_prober.go:28] interesting pod/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Sep 10 22:40:58.738266 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:58.737899 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Sep 10 22:40:59.737321 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:59.737285 2577 patch_prober.go:28] interesting pod/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Sep 10 22:40:59.737499 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:40:59.737341 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Sep 10 22:41:00.741655 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:00.741626 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:41:00.742131 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:00.741912 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:41:00.742732 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:00.742714 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:41:13.173844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.173791 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7577-d3f0e2"] Sep 10 22:41:13.177669 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.177653 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:13.180655 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.180626 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ks5gc\"" Sep 10 22:41:13.190790 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.190762 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7577-d3f0e2"] Sep 10 22:41:13.266388 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.266362 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82cm5\" (UniqueName: \"kubernetes.io/projected/bd92cd1d-65aa-41e4-aea2-e6f809bf888e-kube-api-access-82cm5\") pod \"test-curl-7577-d3f0e2\" (UID: \"bd92cd1d-65aa-41e4-aea2-e6f809bf888e\") " pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:13.367478 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.367448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82cm5\" (UniqueName: \"kubernetes.io/projected/bd92cd1d-65aa-41e4-aea2-e6f809bf888e-kube-api-access-82cm5\") pod \"test-curl-7577-d3f0e2\" (UID: \"bd92cd1d-65aa-41e4-aea2-e6f809bf888e\") " pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:13.377095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.377068 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82cm5\" (UniqueName: \"kubernetes.io/projected/bd92cd1d-65aa-41e4-aea2-e6f809bf888e-kube-api-access-82cm5\") pod \"test-curl-7577-d3f0e2\" (UID: \"bd92cd1d-65aa-41e4-aea2-e6f809bf888e\") " pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:13.487368 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.487292 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:13.772520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.772495 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7577-d3f0e2"] Sep 10 22:41:13.773661 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:41:13.773635 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd92cd1d_65aa_41e4_aea2_e6f809bf888e.slice/crio-1b72c6ee38ac5b4acc9342dd26ebfd220565a71f23e78be292ca71966a8bb999 WatchSource:0}: Error finding container 1b72c6ee38ac5b4acc9342dd26ebfd220565a71f23e78be292ca71966a8bb999: Status 404 returned error can't find the container with id 1b72c6ee38ac5b4acc9342dd26ebfd220565a71f23e78be292ca71966a8bb999 Sep 10 22:41:13.894320 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:13.894294 2577 scope.go:117] "RemoveContainer" containerID="6513fe30c3410a49f46ca6a24ef6e27a44dce61281c7b36f60373e9b5643f789" Sep 10 22:41:14.058979 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:14.058942 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7577-d3f0e2" event={"ID":"bd92cd1d-65aa-41e4-aea2-e6f809bf888e","Type":"ContainerStarted","Data":"8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89"} Sep 10 22:41:14.059116 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:14.058990 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7577-d3f0e2" event={"ID":"bd92cd1d-65aa-41e4-aea2-e6f809bf888e","Type":"ContainerStarted","Data":"1b72c6ee38ac5b4acc9342dd26ebfd220565a71f23e78be292ca71966a8bb999"} Sep 10 22:41:14.078161 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:14.078106 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7577-d3f0e2" podStartSLOduration=1.078081664 podStartE2EDuration="1.078081664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:41:14.075755394 +0000 UTC m=+1380.871970872" watchObservedRunningTime="2026-09-10 22:41:14.078081664 +0000 UTC m=+1380.874297142" Sep 10 22:41:14.707275 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:14.707236 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7577-d3f0e2"] Sep 10 22:41:14.712120 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:14.712085 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7577-d3f0e2"] Sep 10 22:41:15.065257 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:41:15.065227 2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7577-d3f0e2" container="test-curl-7577-d3f0e2" Sep 10 22:41:16.071120 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:16.071080 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7577-d3f0e2" podUID="bd92cd1d-65aa-41e4-aea2-e6f809bf888e" containerName="test-curl-7577-d3f0e2" containerID="cri-o://8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89" gracePeriod=2 Sep 10 22:41:18.267585 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:18.267561 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:18.417273 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:18.417193 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82cm5\" (UniqueName: \"kubernetes.io/projected/bd92cd1d-65aa-41e4-aea2-e6f809bf888e-kube-api-access-82cm5\") pod \"bd92cd1d-65aa-41e4-aea2-e6f809bf888e\" (UID: \"bd92cd1d-65aa-41e4-aea2-e6f809bf888e\") " Sep 10 22:41:18.419325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:18.419292 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd92cd1d-65aa-41e4-aea2-e6f809bf888e-kube-api-access-82cm5" (OuterVolumeSpecName: "kube-api-access-82cm5") pod "bd92cd1d-65aa-41e4-aea2-e6f809bf888e" (UID: "bd92cd1d-65aa-41e4-aea2-e6f809bf888e"). InnerVolumeSpecName "kube-api-access-82cm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:41:18.518833 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:18.518797 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82cm5\" (UniqueName: \"kubernetes.io/projected/bd92cd1d-65aa-41e4-aea2-e6f809bf888e-kube-api-access-82cm5\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:19.098989 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:19.098951 2577 generic.go:356] "Generic (PLEG): container finished" podID="bd92cd1d-65aa-41e4-aea2-e6f809bf888e" containerID="8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89" exitCode=137 Sep 10 22:41:19.098989 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:19.098992 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:19.099233 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:19.099027 2577 scope.go:117] "RemoveContainer" containerID="8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89" Sep 10 22:41:19.109017 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:19.108999 2577 scope.go:117] "RemoveContainer" containerID="8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89" Sep 10 22:41:19.109276 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:41:19.109256 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89\": container with ID starting with 8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89 not found: ID does not exist" containerID="8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89" Sep 10 22:41:19.109358 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:19.109282 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89"} err="failed to get container status \"8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89\": rpc error: code = NotFound desc = could not find container \"8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89\": container with ID starting with 8ad836ce458204f3f011f88350ee4f4b9c10f79bfce15c824f231b799116af89 not found: ID does not exist" Sep 10 22:41:19.109959 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:41:19.109942 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7577-d3f0e2\" is forbidden: User \"system:node:ip-10-0-142-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-235.ec2.internal' and this object" podUID="bd92cd1d-65aa-41e4-aea2-e6f809bf888e" pod="opendatahub/test-curl-7577-d3f0e2" Sep 10 22:41:19.751004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:19.750963 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd92cd1d-65aa-41e4-aea2-e6f809bf888e" path="/var/lib/kubelet/pods/bd92cd1d-65aa-41e4-aea2-e6f809bf888e/volumes" Sep 10 22:41:27.969445 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:27.969413 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p"] Sep 10 22:41:27.969906 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:27.969630 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" containerID="cri-o://fd17bff6eab4dcfb8acfb5c6fbed748e5cbdbcf6adf653d91ef8e4487d26543b" gracePeriod=30 Sep 10 22:41:33.199511 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.199200 2577 generic.go:356] "Generic (PLEG): container finished" podID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerID="fd17bff6eab4dcfb8acfb5c6fbed748e5cbdbcf6adf653d91ef8e4487d26543b" exitCode=0 Sep 10 22:41:33.199511 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.199319 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" event={"ID":"9d8fd23e-55a0-444d-973c-5daee5fd4364","Type":"ContainerDied","Data":"fd17bff6eab4dcfb8acfb5c6fbed748e5cbdbcf6adf653d91ef8e4487d26543b"} Sep 10 22:41:33.199511 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.199381 2577 scope.go:117] "RemoveContainer" containerID="f58c99ff56800ec3b84621e52251ba16fac5822ea50951b7d29db378fecda0d1" Sep 10 22:41:33.233463 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.233440 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:41:33.266499 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266476 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-socket\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266613 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266513 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-envoy\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266613 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266560 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-podinfo\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266688 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266642 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77npw\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-kube-api-access-77npw\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266741 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266706 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-token\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266741 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266736 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-certs\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266767 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-credential-socket\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266844 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-data\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.266923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266870 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d8fd23e-55a0-444d-973c-5daee5fd4364-istiod-ca-cert\") pod \"9d8fd23e-55a0-444d-973c-5daee5fd4364\" (UID: \"9d8fd23e-55a0-444d-973c-5daee5fd4364\") " Sep 10 22:41:33.267639 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.266733 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:33.267639 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.267174 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:33.267639 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.267308 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.267639 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.267315 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:33.267639 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.267330 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.267639 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.267335 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:33.267639 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.267584 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d8fd23e-55a0-444d-973c-5daee5fd4364-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:41:33.270144 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.270107 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:41:33.270237 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.270211 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:33.270501 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.270477 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:41:33.270501 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.270489 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-kube-api-access-77npw" (OuterVolumeSpecName: "kube-api-access-77npw") pod "9d8fd23e-55a0-444d-973c-5daee5fd4364" (UID: "9d8fd23e-55a0-444d-973c-5daee5fd4364"). InnerVolumeSpecName "kube-api-access-77npw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:41:33.368356 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.368329 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.368356 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.368352 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.368356 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.368360 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.368561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.368369 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d8fd23e-55a0-444d-973c-5daee5fd4364-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.368561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.368378 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.368561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.368386 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d8fd23e-55a0-444d-973c-5daee5fd4364-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:33.368561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:33.368394 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77npw\" (UniqueName: \"kubernetes.io/projected/9d8fd23e-55a0-444d-973c-5daee5fd4364-kube-api-access-77npw\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:34.206224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:34.206190 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" event={"ID":"9d8fd23e-55a0-444d-973c-5daee5fd4364","Type":"ContainerDied","Data":"8dad3e42ab2c7a2e4809c444475e388ad7a02a8c18d9487b0eaac00bd912dcb7"} Sep 10 22:41:34.206224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:34.206211 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p" Sep 10 22:41:34.206755 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:34.206260 2577 scope.go:117] "RemoveContainer" containerID="fd17bff6eab4dcfb8acfb5c6fbed748e5cbdbcf6adf653d91ef8e4487d26543b" Sep 10 22:41:34.244078 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:34.244052 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p"] Sep 10 22:41:34.246991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:34.246970 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-03e78e-openshift-default-75d5698cb8-tql5p"] Sep 10 22:41:35.750672 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:35.750637 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" path="/var/lib/kubelet/pods/9d8fd23e-55a0-444d-973c-5daee5fd4364/volumes" Sep 10 22:41:43.463152 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.463114 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9"] Sep 10 22:41:43.463654 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.463347 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" podUID="82b544c3-a4c1-447f-b292-c387f8a0b40a" containerName="maas-api" containerID="cri-o://66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32" gracePeriod=30 Sep 10 22:41:43.721402 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.721339 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:41:43.771112 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.771083 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6zpj\" (UniqueName: \"kubernetes.io/projected/82b544c3-a4c1-447f-b292-c387f8a0b40a-kube-api-access-z6zpj\") pod \"82b544c3-a4c1-447f-b292-c387f8a0b40a\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " Sep 10 22:41:43.771258 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.771134 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-metrics-certs\") pod \"82b544c3-a4c1-447f-b292-c387f8a0b40a\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " Sep 10 22:41:43.771258 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.771191 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls\") pod \"82b544c3-a4c1-447f-b292-c387f8a0b40a\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " Sep 10 22:41:43.771258 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.771214 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82b544c3-a4c1-447f-b292-c387f8a0b40a-openshift-service-ca\") pod \"82b544c3-a4c1-447f-b292-c387f8a0b40a\" (UID: \"82b544c3-a4c1-447f-b292-c387f8a0b40a\") " Sep 10 22:41:43.771671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.771629 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b544c3-a4c1-447f-b292-c387f8a0b40a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82b544c3-a4c1-447f-b292-c387f8a0b40a" (UID: "82b544c3-a4c1-447f-b292-c387f8a0b40a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:41:43.773357 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.773333 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "82b544c3-a4c1-447f-b292-c387f8a0b40a" (UID: "82b544c3-a4c1-447f-b292-c387f8a0b40a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:41:43.773483 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.773427 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82b544c3-a4c1-447f-b292-c387f8a0b40a" (UID: "82b544c3-a4c1-447f-b292-c387f8a0b40a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:41:43.773790 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.773766 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b544c3-a4c1-447f-b292-c387f8a0b40a-kube-api-access-z6zpj" (OuterVolumeSpecName: "kube-api-access-z6zpj") pod "82b544c3-a4c1-447f-b292-c387f8a0b40a" (UID: "82b544c3-a4c1-447f-b292-c387f8a0b40a"). InnerVolumeSpecName "kube-api-access-z6zpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:41:43.873066 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.873033 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6zpj\" (UniqueName: \"kubernetes.io/projected/82b544c3-a4c1-447f-b292-c387f8a0b40a-kube-api-access-z6zpj\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:43.873066 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.873067 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:43.873266 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.873079 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82b544c3-a4c1-447f-b292-c387f8a0b40a-maas-api-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:43.873266 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:43.873101 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82b544c3-a4c1-447f-b292-c387f8a0b40a-openshift-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:44.265383 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.265344 2577 generic.go:356] "Generic (PLEG): container finished" podID="82b544c3-a4c1-447f-b292-c387f8a0b40a" containerID="66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32" exitCode=0 Sep 10 22:41:44.265867 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.265404 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" Sep 10 22:41:44.265867 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.265415 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" event={"ID":"82b544c3-a4c1-447f-b292-c387f8a0b40a","Type":"ContainerDied","Data":"66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32"} Sep 10 22:41:44.265867 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.265495 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9" event={"ID":"82b544c3-a4c1-447f-b292-c387f8a0b40a","Type":"ContainerDied","Data":"056fbea85c383c29736c6230850063da7ba15c0ef41fac8452e61d8133400bc5"} Sep 10 22:41:44.265867 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.265527 2577 scope.go:117] "RemoveContainer" containerID="66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32" Sep 10 22:41:44.276199 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.276182 2577 scope.go:117] "RemoveContainer" containerID="66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32" Sep 10 22:41:44.276457 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:41:44.276440 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32\": container with ID starting with 66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32 not found: ID does not exist" containerID="66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32" Sep 10 22:41:44.276509 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.276465 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32"} err="failed to get container status \"66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32\": rpc error: code = NotFound desc = could not find container \"66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32\": container with ID starting with 66861958a336a5a6e5264919fe8aaccfe9fe2a91dc25e354f8e5147907faec32 not found: ID does not exist" Sep 10 22:41:44.288449 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.288424 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9"] Sep 10 22:41:44.296252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:44.296225 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ecb38e-5d57f6848b-fr2w9"] Sep 10 22:41:45.750315 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:45.750286 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b544c3-a4c1-447f-b292-c387f8a0b40a" path="/var/lib/kubelet/pods/82b544c3-a4c1-447f-b292-c387f8a0b40a/volumes" Sep 10 22:41:50.016682 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:50.016643 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r"] Sep 10 22:41:50.017088 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:50.016924 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" containerID="cri-o://b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31" gracePeriod=30 Sep 10 22:41:55.283633 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.283576 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:41:55.337245 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.337210 2577 generic.go:356] "Generic (PLEG): container finished" podID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerID="b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31" exitCode=0 Sep 10 22:41:55.337411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.337268 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" Sep 10 22:41:55.337411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.337280 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" event={"ID":"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce","Type":"ContainerDied","Data":"b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31"} Sep 10 22:41:55.337411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.337320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r" event={"ID":"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce","Type":"ContainerDied","Data":"f73716b7a72a82068fb32ad81599b1782a879f9fdc1cb1b01339696db956a71c"} Sep 10 22:41:55.337411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.337345 2577 scope.go:117] "RemoveContainer" containerID="b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31" Sep 10 22:41:55.347641 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.347615 2577 scope.go:117] "RemoveContainer" containerID="52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b" Sep 10 22:41:55.356007 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.355989 2577 scope.go:117] "RemoveContainer" containerID="b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31" Sep 10 22:41:55.356262 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:41:55.356242 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31\": container with ID starting with b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31 not found: ID does not exist" containerID="b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31" Sep 10 22:41:55.356316 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.356272 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31"} err="failed to get container status \"b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31\": rpc error: code = NotFound desc = could not find container \"b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31\": container with ID starting with b44aab8997cc7c8d2985f57ee06907c4c4244d96b19a7c96122f5077948efc31 not found: ID does not exist" Sep 10 22:41:55.356316 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.356289 2577 scope.go:117] "RemoveContainer" containerID="52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b" Sep 10 22:41:55.356574 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:41:55.356558 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b\": container with ID starting with 52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b not found: ID does not exist" containerID="52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b" Sep 10 22:41:55.356611 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.356579 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b"} err="failed to get container status \"52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b\": rpc error: code = NotFound desc = could not find container \"52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b\": container with ID starting with 52cd4d65523c2beaa75122b3044483afc1a5130e241bca0396294759d2bf0a7b not found: ID does not exist" Sep 10 22:41:55.379957 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.379931 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpfsb\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-kube-api-access-kpfsb\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380072 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.379976 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-data\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380072 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.379992 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istiod-ca-cert\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380072 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380031 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-credential-socket\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380072 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380055 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-socket\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380074 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-envoy\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380140 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-certs\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380246 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-podinfo\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380261 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:55.380290 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380280 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-token\") pod \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\" (UID: \"bd3d0d05-a2af-424a-a8f5-b3d62721f1ce\") " Sep 10 22:41:55.380553 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380351 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:55.380553 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380489 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:55.380553 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380534 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:41:55.380715 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380666 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:55.380781 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380734 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.380781 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380754 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.380781 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380768 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.380931 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.380791 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.382494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.382473 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:41:55.382593 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.382498 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:41:55.382784 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.382766 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-kube-api-access-kpfsb" (OuterVolumeSpecName: "kube-api-access-kpfsb") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "kube-api-access-kpfsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:41:55.382864 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.382811 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" (UID: "bd3d0d05-a2af-424a-a8f5-b3d62721f1ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:41:55.481586 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.481552 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.481586 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.481580 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.481586 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.481592 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.481847 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.481603 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.481847 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.481617 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpfsb\" (UniqueName: \"kubernetes.io/projected/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce-kube-api-access-kpfsb\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:41:55.663599 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.663557 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r"] Sep 10 22:41:55.667187 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.667161 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ecb38e-openshift-default-58d4d7986-7fz6r"] Sep 10 22:41:55.750273 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:41:55.750239 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" path="/var/lib/kubelet/pods/bd3d0d05-a2af-424a-a8f5-b3d62721f1ce/volumes" Sep 10 22:42:01.108046 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.108013 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb"] Sep 10 22:42:01.108596 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.108250 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" podUID="8e36069b-3d33-4cb5-9c91-be1cd37e032b" containerName="maas-api" containerID="cri-o://3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680" gracePeriod=30 Sep 10 22:42:01.357270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.357240 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:42:01.378671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.378530 2577 generic.go:356] "Generic (PLEG): container finished" podID="8e36069b-3d33-4cb5-9c91-be1cd37e032b" containerID="3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680" exitCode=0 Sep 10 22:42:01.378671 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.378644 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" event={"ID":"8e36069b-3d33-4cb5-9c91-be1cd37e032b","Type":"ContainerDied","Data":"3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680"} Sep 10 22:42:01.378946 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.378749 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" event={"ID":"8e36069b-3d33-4cb5-9c91-be1cd37e032b","Type":"ContainerDied","Data":"bbbb4f43e476b57355c28452ba6c886b3b52dc0923da30f20bd74d46fe41bd7b"} Sep 10 22:42:01.378946 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.378781 2577 scope.go:117] "RemoveContainer" containerID="3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680" Sep 10 22:42:01.379901 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.379267 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb" Sep 10 22:42:01.391645 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.391626 2577 scope.go:117] "RemoveContainer" containerID="3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680" Sep 10 22:42:01.392020 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:42:01.392000 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680\": container with ID starting with 3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680 not found: ID does not exist" containerID="3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680" Sep 10 22:42:01.392095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.392031 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680"} err="failed to get container status \"3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680\": rpc error: code = NotFound desc = could not find container \"3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680\": container with ID starting with 3c0db78db6c2b7c740586f5b65894ae8516b3c73193240f066cc7f1ad2ac7680 not found: ID does not exist" Sep 10 22:42:01.537963 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.537935 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e36069b-3d33-4cb5-9c91-be1cd37e032b-openshift-service-ca\") pod \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " Sep 10 22:42:01.538138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.537974 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdm7s\" (UniqueName: \"kubernetes.io/projected/8e36069b-3d33-4cb5-9c91-be1cd37e032b-kube-api-access-bdm7s\") pod \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " Sep 10 22:42:01.538138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.538055 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-metrics-certs\") pod \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " Sep 10 22:42:01.538138 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.538136 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls\") pod \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\" (UID: \"8e36069b-3d33-4cb5-9c91-be1cd37e032b\") " Sep 10 22:42:01.538341 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.538310 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e36069b-3d33-4cb5-9c91-be1cd37e032b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8e36069b-3d33-4cb5-9c91-be1cd37e032b" (UID: "8e36069b-3d33-4cb5-9c91-be1cd37e032b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:42:01.538419 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.538395 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e36069b-3d33-4cb5-9c91-be1cd37e032b-openshift-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:01.540361 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.540325 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8e36069b-3d33-4cb5-9c91-be1cd37e032b" (UID: "8e36069b-3d33-4cb5-9c91-be1cd37e032b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:42:01.540361 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.540336 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8e36069b-3d33-4cb5-9c91-be1cd37e032b" (UID: "8e36069b-3d33-4cb5-9c91-be1cd37e032b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:42:01.540556 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.540534 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e36069b-3d33-4cb5-9c91-be1cd37e032b-kube-api-access-bdm7s" (OuterVolumeSpecName: "kube-api-access-bdm7s") pod "8e36069b-3d33-4cb5-9c91-be1cd37e032b" (UID: "8e36069b-3d33-4cb5-9c91-be1cd37e032b"). InnerVolumeSpecName "kube-api-access-bdm7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:42:01.639230 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.639144 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:01.639230 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.639175 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e36069b-3d33-4cb5-9c91-be1cd37e032b-maas-api-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:01.639230 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.639184 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdm7s\" (UniqueName: \"kubernetes.io/projected/8e36069b-3d33-4cb5-9c91-be1cd37e032b-kube-api-access-bdm7s\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:01.704493 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.704462 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb"] Sep 10 22:42:01.711219 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.711191 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-7a7e94-84cf568cc7-hpbnb"] Sep 10 22:42:01.750169 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:01.750128 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e36069b-3d33-4cb5-9c91-be1cd37e032b" path="/var/lib/kubelet/pods/8e36069b-3d33-4cb5-9c91-be1cd37e032b/volumes" Sep 10 22:42:25.760705 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:25.760673 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr"] Sep 10 22:42:25.761261 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:25.760895 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" containerID="cri-o://8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784" gracePeriod=30 Sep 10 22:42:26.001128 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:26.001086 2577 patch_prober.go:28] interesting pod/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 22:42:26.001333 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:26.001173 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 22:42:31.016735 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.016712 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:42:31.104294 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104233 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-credential-socket\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104294 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104264 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-envoy\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104294 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104292 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-data\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104523 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104332 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-socket\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104523 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104370 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abfb18a5-4300-4dd3-afc7-e04309ac695f-istiod-ca-cert\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104523 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104407 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-certs\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104523 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104521 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-token\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104751 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104522 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:42:31.104751 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104546 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r845d\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-kube-api-access-r845d\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104751 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104584 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-podinfo\") pod \"abfb18a5-4300-4dd3-afc7-e04309ac695f\" (UID: \"abfb18a5-4300-4dd3-afc7-e04309ac695f\") " Sep 10 22:42:31.104751 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104599 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-data" (OuterVolumeSpecName: "istio-data") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:42:31.104999 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104784 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:42:31.104999 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104901 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:42:31.104999 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104955 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.104999 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104976 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.104999 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.104993 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.105241 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.105145 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abfb18a5-4300-4dd3-afc7-e04309ac695f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:42:31.106529 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.106504 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:42:31.106968 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.106951 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-token" (OuterVolumeSpecName: "istio-token") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:42:31.107182 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.107169 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:42:31.107271 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.107251 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-kube-api-access-r845d" (OuterVolumeSpecName: "kube-api-access-r845d") pod "abfb18a5-4300-4dd3-afc7-e04309ac695f" (UID: "abfb18a5-4300-4dd3-afc7-e04309ac695f"). InnerVolumeSpecName "kube-api-access-r845d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:42:31.206384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.206361 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.206384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.206385 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abfb18a5-4300-4dd3-afc7-e04309ac695f-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.206529 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.206395 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abfb18a5-4300-4dd3-afc7-e04309ac695f-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.206529 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.206403 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.206529 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.206411 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r845d\" (UniqueName: \"kubernetes.io/projected/abfb18a5-4300-4dd3-afc7-e04309ac695f-kube-api-access-r845d\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.206529 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.206420 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abfb18a5-4300-4dd3-afc7-e04309ac695f-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:42:31.573781 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.573748 2577 generic.go:356] "Generic (PLEG): container finished" podID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerID="8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784" exitCode=0 Sep 10 22:42:31.573957 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.573797 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" Sep 10 22:42:31.573957 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.573839 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" event={"ID":"abfb18a5-4300-4dd3-afc7-e04309ac695f","Type":"ContainerDied","Data":"8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784"} Sep 10 22:42:31.573957 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.573890 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr" event={"ID":"abfb18a5-4300-4dd3-afc7-e04309ac695f","Type":"ContainerDied","Data":"4a5be1d5423b5103dbeae862b05d25864e7dc6a2115d0ebe3d5c4e4573ca5372"} Sep 10 22:42:31.573957 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.573912 2577 scope.go:117] "RemoveContainer" containerID="8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784" Sep 10 22:42:31.583702 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.583685 2577 scope.go:117] "RemoveContainer" containerID="5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b" Sep 10 22:42:31.594302 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.594281 2577 scope.go:117] "RemoveContainer" containerID="8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784" Sep 10 22:42:31.594589 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:42:31.594570 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784\": container with ID starting with 8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784 not found: ID does not exist" containerID="8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784" Sep 10 22:42:31.594644 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.594598 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784"} err="failed to get container status \"8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784\": rpc error: code = NotFound desc = could not find container \"8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784\": container with ID starting with 8599cbd7e11114a6f7c750f37dc3007e6b8301326a96484f34cb67b0dd80e784 not found: ID does not exist" Sep 10 22:42:31.594644 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.594614 2577 scope.go:117] "RemoveContainer" containerID="5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b" Sep 10 22:42:31.594942 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:42:31.594917 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b\": container with ID starting with 5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b not found: ID does not exist" containerID="5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b" Sep 10 22:42:31.595015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.594951 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b"} err="failed to get container status \"5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b\": rpc error: code = NotFound desc = could not find container \"5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b\": container with ID starting with 5dc1f6ea048d668fde592da3b03615cbe2e40c8c4777124c3f9e6bc2eecc2a8b not found: ID does not exist" Sep 10 22:42:31.596220 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.596202 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr"] Sep 10 22:42:31.600289 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.600266 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-25e5bd-openshift-default-6d699865c4-nlqqr"] Sep 10 22:42:31.754837 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:31.754794 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" path="/var/lib/kubelet/pods/abfb18a5-4300-4dd3-afc7-e04309ac695f/volumes" Sep 10 22:42:54.680801 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.680765 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf"] Sep 10 22:42:54.681446 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681426 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" Sep 10 22:42:54.681488 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681452 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" Sep 10 22:42:54.681528 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681485 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" Sep 10 22:42:54.681528 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681495 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" Sep 10 22:42:54.681592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681529 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e36069b-3d33-4cb5-9c91-be1cd37e032b" containerName="maas-api" Sep 10 22:42:54.681592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681540 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e36069b-3d33-4cb5-9c91-be1cd37e032b" containerName="maas-api" Sep 10 22:42:54.681592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681552 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd92cd1d-65aa-41e4-aea2-e6f809bf888e" containerName="test-curl-7577-d3f0e2" Sep 10 22:42:54.681592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681560 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd92cd1d-65aa-41e4-aea2-e6f809bf888e" containerName="test-curl-7577-d3f0e2" Sep 10 22:42:54.681592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681573 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" Sep 10 22:42:54.681592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681581 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" Sep 10 22:42:54.681592 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681591 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681599 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681615 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681624 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681632 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b544c3-a4c1-447f-b292-c387f8a0b40a" containerName="maas-api" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681649 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b544c3-a4c1-447f-b292-c387f8a0b40a" containerName="maas-api" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681831 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681844 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681855 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="abfb18a5-4300-4dd3-afc7-e04309ac695f" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681864 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681874 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd92cd1d-65aa-41e4-aea2-e6f809bf888e" containerName="test-curl-7577-d3f0e2" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681885 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e36069b-3d33-4cb5-9c91-be1cd37e032b" containerName="maas-api" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681894 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681904 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d8fd23e-55a0-444d-973c-5daee5fd4364" containerName="istio-proxy" Sep 10 22:42:54.681970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.681912 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b544c3-a4c1-447f-b292-c387f8a0b40a" containerName="maas-api" Sep 10 22:42:54.685543 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.685514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.687934 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.687915 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-16d0fd-openshift-default-dockercfg-2fnvq\"" Sep 10 22:42:54.695859 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.695834 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf"] Sep 10 22:42:54.812784 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.812747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddab46d-eebf-43d4-937f-f29835588772-istio-podinfo\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.812797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-envoy\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.812939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddab46d-eebf-43d4-937f-f29835588772-istiod-ca-cert\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813004 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.813000 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-certs\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.813034 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-data\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.813091 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-socket\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.813109 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-istio-token\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.813141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-credential-socket\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.813224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.813158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77wc\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-kube-api-access-s77wc\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.913769 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.913727 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddab46d-eebf-43d4-937f-f29835588772-istio-podinfo\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.913978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.913777 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-envoy\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.913978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.913863 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddab46d-eebf-43d4-937f-f29835588772-istiod-ca-cert\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.913978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.913917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-certs\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.913978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.913945 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-data\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914188 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-socket\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914188 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914060 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-istio-token\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914188 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-credential-socket\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914188 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914151 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77wc\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-kube-api-access-s77wc\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914413 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914387 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-certs\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914480 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914426 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-data\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914624 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-credential-socket\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914695 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914653 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-socket\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.914912 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.914894 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddab46d-eebf-43d4-937f-f29835588772-istiod-ca-cert\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.916991 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.916970 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-envoy\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.917261 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.917240 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddab46d-eebf-43d4-937f-f29835588772-istio-podinfo\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.921658 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.921636 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-istio-token\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.921993 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.921973 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77wc\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-kube-api-access-s77wc\") pod \"e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:54.996918 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:54.996851 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:55.279018 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.278989 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf"] Sep 10 22:42:55.280887 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:42:55.280860 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ddab46d_eebf_43d4_937f_f29835588772.slice/crio-9f93677d993c8bd12f018b1fcb3e874717b325da9098b0cf6696a65d74c2ce6c WatchSource:0}: Error finding container 9f93677d993c8bd12f018b1fcb3e874717b325da9098b0cf6696a65d74c2ce6c: Status 404 returned error can't find the container with id 9f93677d993c8bd12f018b1fcb3e874717b325da9098b0cf6696a65d74c2ce6c Sep 10 22:42:55.283328 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.283287 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:42:55.283421 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.283378 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:42:55.283465 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.283423 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:42:55.719915 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.719797 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" event={"ID":"0ddab46d-eebf-43d4-937f-f29835588772","Type":"ContainerStarted","Data":"9c59041463ee9e89d1d4e8d823a5082239b061934179cd1931a14ffe8e447922"} Sep 10 22:42:55.719915 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.719871 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" event={"ID":"0ddab46d-eebf-43d4-937f-f29835588772","Type":"ContainerStarted","Data":"9f93677d993c8bd12f018b1fcb3e874717b325da9098b0cf6696a65d74c2ce6c"} Sep 10 22:42:55.740268 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.740217 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podStartSLOduration=1.740202724 podStartE2EDuration="1.740202724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:42:55.737583462 +0000 UTC m=+1482.533798952" watchObservedRunningTime="2026-09-10 22:42:55.740202724 +0000 UTC m=+1482.536418200" Sep 10 22:42:55.997065 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:55.996978 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:57.002895 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:57.002861 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:57.731745 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:57.731712 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:42:57.732648 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:42:57.732625 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:43:13.774973 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:13.774937 2577 patch_prober.go:28] interesting pod/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 22:43:13.775444 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:13.775004 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 22:43:13.794929 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:13.794905 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:43:13.802812 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:13.802790 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:43:14.844533 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:14.844494 2577 generic.go:356] "Generic (PLEG): container finished" podID="0ddab46d-eebf-43d4-937f-f29835588772" containerID="9c59041463ee9e89d1d4e8d823a5082239b061934179cd1931a14ffe8e447922" exitCode=0 Sep 10 22:43:14.844966 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:14.844568 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" event={"ID":"0ddab46d-eebf-43d4-937f-f29835588772","Type":"ContainerDied","Data":"9c59041463ee9e89d1d4e8d823a5082239b061934179cd1931a14ffe8e447922"} Sep 10 22:43:14.845022 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:14.844992 2577 scope.go:117] "RemoveContainer" containerID="9c59041463ee9e89d1d4e8d823a5082239b061934179cd1931a14ffe8e447922" Sep 10 22:43:14.846101 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:14.846076 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:14.846170 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:14.846147 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:14.846210 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:14.846177 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:15.852595 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:15.852563 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" event={"ID":"0ddab46d-eebf-43d4-937f-f29835588772","Type":"ContainerStarted","Data":"033ae96a371adec2fd49c97b5a5ab43555d79594bfbbf4ddde01207bdcaafed4"} Sep 10 22:43:15.997784 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:15.997747 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:43:15.998105 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:15.998069 2577 patch_prober.go:28] interesting pod/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Sep 10 22:43:15.998212 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:15.998139 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Sep 10 22:43:16.998215 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:16.998167 2577 patch_prober.go:28] interesting pod/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Sep 10 22:43:16.998703 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:16.998240 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Sep 10 22:43:17.998228 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:17.998183 2577 patch_prober.go:28] interesting pod/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Sep 10 22:43:17.998718 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:17.998252 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Sep 10 22:43:18.997596 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:18.997552 2577 patch_prober.go:28] interesting pod/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Sep 10 22:43:18.997808 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:18.997628 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Sep 10 22:43:19.997806 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:19.997770 2577 patch_prober.go:28] interesting pod/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Sep 10 22:43:19.998410 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:19.998359 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Sep 10 22:43:21.039257 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:21.039225 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:43:21.039647 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:21.039534 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:43:21.040257 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:21.040237 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:43:23.675202 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.675170 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s"] Sep 10 22:43:23.675748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.675715 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" Sep 10 22:43:23.675748 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.675737 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3d0d05-a2af-424a-a8f5-b3d62721f1ce" containerName="istio-proxy" Sep 10 22:43:23.678133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.678117 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.680908 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.680883 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-d28770-openshift-default-dockercfg-6vzzf\"" Sep 10 22:43:23.726713 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.726683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s"] Sep 10 22:43:23.785684 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.785649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-socket\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.785997 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.785975 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-certs\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.786329 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.786310 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb8d4\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-kube-api-access-mb8d4\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.786786 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.786727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-podinfo\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.787177 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.787158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-credential-socket\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.787421 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.787402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istiod-ca-cert\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.787610 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.787583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-data\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.787760 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.787738 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-token\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.787920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.787802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-envoy\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istiod-ca-cert\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888594 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-data\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-token\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-envoy\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888690 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-socket\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-certs\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb8d4\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-kube-api-access-mb8d4\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888801 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-podinfo\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.888891 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.888857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-credential-socket\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.889252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.889160 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-socket\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.889252 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.889239 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-certs\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.889438 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.889359 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-data\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.889504 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.889442 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-credential-socket\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.889504 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.889469 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istiod-ca-cert\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.891368 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.891341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-envoy\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.891831 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.891796 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-podinfo\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.903149 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.903126 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-token\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.903352 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.903332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb8d4\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-kube-api-access-mb8d4\") pod \"e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:23.988756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:23.988685 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:24.269030 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.268998 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s"] Sep 10 22:43:24.270446 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:43:24.270411 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12cb822_61a6_44e7_b0aa_404fac00dc0d.slice/crio-73dc7c7be49f0ee091a477a0365ee696c4837a062f4171beca0d14745be8ec52 WatchSource:0}: Error finding container 73dc7c7be49f0ee091a477a0365ee696c4837a062f4171beca0d14745be8ec52: Status 404 returned error can't find the container with id 73dc7c7be49f0ee091a477a0365ee696c4837a062f4171beca0d14745be8ec52 Sep 10 22:43:24.272560 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.272529 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:24.272650 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.272586 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:24.272650 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.272614 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:24.916108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.916073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" event={"ID":"c12cb822-61a6-44e7-b0aa-404fac00dc0d","Type":"ContainerStarted","Data":"18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085"} Sep 10 22:43:24.916487 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.916159 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" event={"ID":"c12cb822-61a6-44e7-b0aa-404fac00dc0d","Type":"ContainerStarted","Data":"73dc7c7be49f0ee091a477a0365ee696c4837a062f4171beca0d14745be8ec52"} Sep 10 22:43:24.935483 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.935422 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podStartSLOduration=1.935407882 podStartE2EDuration="1.935407882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:43:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:43:24.933890332 +0000 UTC m=+1511.730105811" watchObservedRunningTime="2026-09-10 22:43:24.935407882 +0000 UTC m=+1511.731623380" Sep 10 22:43:24.989155 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:24.989120 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:25.994451 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:25.994419 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:26.927080 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:26.927049 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:26.928246 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:26.928228 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:27.973357 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:27.973324 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm"] Sep 10 22:43:27.976759 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:27.976738 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:27.979123 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:27.979101 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-d28770-serving-cert\"" Sep 10 22:43:27.989522 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:27.989495 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm"] Sep 10 22:43:28.033631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.033602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w54k6\" (UniqueName: \"kubernetes.io/projected/074e850d-3c45-49f7-9be1-d56fd2546be8-kube-api-access-w54k6\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.033807 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.033733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.033922 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.033839 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074e850d-3c45-49f7-9be1-d56fd2546be8-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.033922 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.033866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-metrics-certs\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.134730 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.134690 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.134730 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.134734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074e850d-3c45-49f7-9be1-d56fd2546be8-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.135028 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.134757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-metrics-certs\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.135028 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.134794 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w54k6\" (UniqueName: \"kubernetes.io/projected/074e850d-3c45-49f7-9be1-d56fd2546be8-kube-api-access-w54k6\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.135028 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:43:28.134875 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-serving-cert: secret "maas-api-e2e-ipp-b-d28770-serving-cert" not found Sep 10 22:43:28.135028 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:43:28.134961 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls podName:074e850d-3c45-49f7-9be1-d56fd2546be8 nodeName:}" failed. No retries permitted until 2026-09-10 22:43:28.634944429 +0000 UTC m=+1515.431159908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls") pod "maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" (UID: "074e850d-3c45-49f7-9be1-d56fd2546be8") : secret "maas-api-e2e-ipp-b-d28770-serving-cert" not found Sep 10 22:43:28.135336 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.135318 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074e850d-3c45-49f7-9be1-d56fd2546be8-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.137434 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.137412 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-metrics-certs\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.143221 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.143199 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w54k6\" (UniqueName: \"kubernetes.io/projected/074e850d-3c45-49f7-9be1-d56fd2546be8-kube-api-access-w54k6\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.640423 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.640384 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.642872 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.642847 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls\") pod \"maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:28.891182 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:28.891096 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:29.173970 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:29.173944 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm"] Sep 10 22:43:29.175591 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:43:29.175557 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074e850d_3c45_49f7_9be1_d56fd2546be8.slice/crio-0dbcdea628f1c7f92ef9df3de814ca6fed9a683918780465239741bddc7e4ced WatchSource:0}: Error finding container 0dbcdea628f1c7f92ef9df3de814ca6fed9a683918780465239741bddc7e4ced: Status 404 returned error can't find the container with id 0dbcdea628f1c7f92ef9df3de814ca6fed9a683918780465239741bddc7e4ced Sep 10 22:43:29.177333 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:29.177312 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:43:29.947346 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:29.947308 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" event={"ID":"074e850d-3c45-49f7-9be1-d56fd2546be8","Type":"ContainerStarted","Data":"e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b"} Sep 10 22:43:29.947558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:29.947383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" event={"ID":"074e850d-3c45-49f7-9be1-d56fd2546be8","Type":"ContainerStarted","Data":"0dbcdea628f1c7f92ef9df3de814ca6fed9a683918780465239741bddc7e4ced"} Sep 10 22:43:29.947558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:29.947412 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:29.965511 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:29.965465 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" podStartSLOduration=2.725553837 podStartE2EDuration="2.965450152s" totalImagesPullingTime="239.896315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:43:29.962523681 +0000 UTC m=+1516.758739160" watchObservedRunningTime="2026-09-10 22:43:29.965450152 +0000 UTC m=+1516.761665629" Sep 10 22:43:35.959384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:35.959296 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:43:42.967981 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:42.967939 2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 22:43:42.968525 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:42.968015 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 22:43:44.041902 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:44.041864 2577 generic.go:356] "Generic (PLEG): container finished" podID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerID="18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085" exitCode=0 Sep 10 22:43:44.042325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:44.041909 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" event={"ID":"c12cb822-61a6-44e7-b0aa-404fac00dc0d","Type":"ContainerDied","Data":"18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085"} Sep 10 22:43:44.042374 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:44.042355 2577 scope.go:117] "RemoveContainer" containerID="18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085" Sep 10 22:43:44.043469 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:44.043443 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:44.043525 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:44.043510 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:44.043569 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:44.043540 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"} Sep 10 22:43:45.050160 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:45.050125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" event={"ID":"c12cb822-61a6-44e7-b0aa-404fac00dc0d","Type":"ContainerStarted","Data":"452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916"} Sep 10 22:43:45.989554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:45.989517 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:45.989758 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:45.989714 2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Sep 10 22:43:45.989857 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:45.989775 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Sep 10 22:43:46.989549 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:46.989511 2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Sep 10 22:43:46.990189 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:46.990154 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Sep 10 22:43:47.989343 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:47.989305 2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Sep 10 22:43:47.989634 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:47.989600 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Sep 10 22:43:48.989206 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:48.989161 2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Sep 10 22:43:48.989420 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:48.989230 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Sep 10 22:43:49.989785 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:49.989750 2577 patch_prober.go:28] interesting pod/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Sep 10 22:43:49.990192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:49.989834 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Sep 10 22:43:50.993465 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:50.993432 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:50.993863 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:50.993765 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:50.994405 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:50.994387 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:43:56.791227 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:43:56.791200 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t7dns_471c8294-5aad-47d3-8392-d937af251324/payload-processing/0.log" Sep 10 22:44:37.713452 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:44:37.713417 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t7dns_471c8294-5aad-47d3-8392-d937af251324/payload-processing/0.log" Sep 10 22:45:00.139576 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.139534 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z"] Sep 10 22:45:00.142381 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.142365 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p"] Sep 10 22:45:00.142502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.142481 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z" Sep 10 22:45:00.144969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.144927 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p" Sep 10 22:45:00.145207 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.145190 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bltkc\"" Sep 10 22:45:00.156267 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.156242 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z"] Sep 10 22:45:00.156538 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.156514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z" Sep 10 22:45:00.164553 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.164298 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p"] Sep 10 22:45:00.168111 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.167331 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p" Sep 10 22:45:00.451643 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.450372 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z"] Sep 10 22:45:00.451800 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:45:00.451708 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbcf03be_708f_4375_a31e_9de06f6a9372.slice/crio-1ae1493d31a4582d863efe78e47c49ed1f99a4c34d0a90da3951416808e77f55 WatchSource:0}: Error finding container 1ae1493d31a4582d863efe78e47c49ed1f99a4c34d0a90da3951416808e77f55: Status 404 returned error can't find the container with id 1ae1493d31a4582d863efe78e47c49ed1f99a4c34d0a90da3951416808e77f55 Sep 10 22:45:00.454107 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.454080 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p"] Sep 10 22:45:00.454483 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:45:00.454455 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c83503_367e_43c6_82b1_32ef9596d563.slice/crio-c7e9a5f2f2f3ccf2c2b3d32075a024d204f77ebaa5b743bd3a056bb59ee83892 WatchSource:0}: Error finding container c7e9a5f2f2f3ccf2c2b3d32075a024d204f77ebaa5b743bd3a056bb59ee83892: Status 404 returned error can't find the container with id c7e9a5f2f2f3ccf2c2b3d32075a024d204f77ebaa5b743bd3a056bb59ee83892 Sep 10 22:45:00.508108 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.508080 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z" event={"ID":"bbcf03be-708f-4375-a31e-9de06f6a9372","Type":"ContainerStarted","Data":"1ae1493d31a4582d863efe78e47c49ed1f99a4c34d0a90da3951416808e77f55"} Sep 10 22:45:00.509576 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:00.509555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p" event={"ID":"26c83503-367e-43c6-82b1-32ef9596d563","Type":"ContainerStarted","Data":"c7e9a5f2f2f3ccf2c2b3d32075a024d204f77ebaa5b743bd3a056bb59ee83892"} Sep 10 22:45:01.517119 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:01.517088 2577 generic.go:356] "Generic (PLEG): container finished" podID="bbcf03be-708f-4375-a31e-9de06f6a9372" containerID="41f0d8be0224d7f8866f8f282261cc4a2a22bbd99c145b29bf9f7d66bf91e2e4" exitCode=0 Sep 10 22:45:01.517455 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:01.517171 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z" event={"ID":"bbcf03be-708f-4375-a31e-9de06f6a9372","Type":"ContainerDied","Data":"41f0d8be0224d7f8866f8f282261cc4a2a22bbd99c145b29bf9f7d66bf91e2e4"} Sep 10 22:45:01.518618 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:01.518595 2577 generic.go:356] "Generic (PLEG): container finished" podID="26c83503-367e-43c6-82b1-32ef9596d563" containerID="339aa8f3e8410778384347f2f13a8755357084c13aa4382a3aafec4e45f10986" exitCode=0 Sep 10 22:45:01.518715 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:01.518627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p" event={"ID":"26c83503-367e-43c6-82b1-32ef9596d563","Type":"ContainerDied","Data":"339aa8f3e8410778384347f2f13a8755357084c13aa4382a3aafec4e45f10986"} Sep 10 22:45:02.681115 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:02.681084 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p" Sep 10 22:45:02.684285 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:02.684260 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z" Sep 10 22:45:03.532857 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:03.532830 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z" Sep 10 22:45:03.533076 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:03.532859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z" event={"ID":"bbcf03be-708f-4375-a31e-9de06f6a9372","Type":"ContainerDied","Data":"1ae1493d31a4582d863efe78e47c49ed1f99a4c34d0a90da3951416808e77f55"} Sep 10 22:45:03.533076 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:03.532948 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ae1493d31a4582d863efe78e47c49ed1f99a4c34d0a90da3951416808e77f55" Sep 10 22:45:03.534363 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:03.534338 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p" event={"ID":"26c83503-367e-43c6-82b1-32ef9596d563","Type":"ContainerDied","Data":"c7e9a5f2f2f3ccf2c2b3d32075a024d204f77ebaa5b743bd3a056bb59ee83892"} Sep 10 22:45:03.534474 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:03.534366 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p" Sep 10 22:45:03.534474 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:03.534377 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7e9a5f2f2f3ccf2c2b3d32075a024d204f77ebaa5b743bd3a056bb59ee83892" Sep 10 22:45:39.418318 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:39.418223 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z"] Sep 10 22:45:39.421283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:39.421251 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-16d0fd-29818005-wfn8z"] Sep 10 22:45:39.753952 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:45:39.753859 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbcf03be-708f-4375-a31e-9de06f6a9372" path="/var/lib/kubelet/pods/bbcf03be-708f-4375-a31e-9de06f6a9372/volumes" Sep 10 22:46:33.958129 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:33.958097 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf"] Sep 10 22:46:33.958660 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:33.958308 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" containerID="cri-o://033ae96a371adec2fd49c97b5a5ab43555d79594bfbbf4ddde01207bdcaafed4" gracePeriod=30 Sep 10 22:46:36.041739 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:36.041662 2577 patch_prober.go:28] interesting pod/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 22:46:36.042103 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:36.041730 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 22:46:39.114562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.114526 2577 generic.go:356] "Generic (PLEG): container finished" podID="0ddab46d-eebf-43d4-937f-f29835588772" containerID="033ae96a371adec2fd49c97b5a5ab43555d79594bfbbf4ddde01207bdcaafed4" exitCode=0 Sep 10 22:46:39.115011 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.114574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" event={"ID":"0ddab46d-eebf-43d4-937f-f29835588772","Type":"ContainerDied","Data":"033ae96a371adec2fd49c97b5a5ab43555d79594bfbbf4ddde01207bdcaafed4"} Sep 10 22:46:39.115011 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.114627 2577 scope.go:117] "RemoveContainer" containerID="9c59041463ee9e89d1d4e8d823a5082239b061934179cd1931a14ffe8e447922" Sep 10 22:46:39.222893 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.222801 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:46:39.255725 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255676 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-istio-token\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.255725 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255724 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77wc\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-kube-api-access-s77wc\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.255956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255764 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddab46d-eebf-43d4-937f-f29835588772-istio-podinfo\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.255956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255800 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-certs\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.255956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255861 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-credential-socket\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.255956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255893 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-envoy\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.255956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255916 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddab46d-eebf-43d4-937f-f29835588772-istiod-ca-cert\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.255956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.255951 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-socket\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.256270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.256065 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-data\") pod \"0ddab46d-eebf-43d4-937f-f29835588772\" (UID: \"0ddab46d-eebf-43d4-937f-f29835588772\") " Sep 10 22:46:39.256270 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.256231 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:39.256430 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.256408 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:39.256491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.256436 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.256778 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.256605 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:39.256987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.256951 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:39.257254 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.257026 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ddab46d-eebf-43d4-937f-f29835588772-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:46:39.259272 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.259228 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:39.259574 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.259547 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-kube-api-access-s77wc" (OuterVolumeSpecName: "kube-api-access-s77wc") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "kube-api-access-s77wc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:46:39.259681 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.259659 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ddab46d-eebf-43d4-937f-f29835588772-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:46:39.259681 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.259674 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ddab46d-eebf-43d4-937f-f29835588772" (UID: "0ddab46d-eebf-43d4-937f-f29835588772"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:46:39.356992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.356968 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.356992 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.356992 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77wc\" (UniqueName: \"kubernetes.io/projected/0ddab46d-eebf-43d4-937f-f29835588772-kube-api-access-s77wc\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.357133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.357003 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ddab46d-eebf-43d4-937f-f29835588772-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.357133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.357012 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.357133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.357020 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.357133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.357027 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ddab46d-eebf-43d4-937f-f29835588772-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.357133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.357035 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:39.357133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:39.357042 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ddab46d-eebf-43d4-937f-f29835588772-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:40.122510 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:40.122473 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" event={"ID":"0ddab46d-eebf-43d4-937f-f29835588772","Type":"ContainerDied","Data":"9f93677d993c8bd12f018b1fcb3e874717b325da9098b0cf6696a65d74c2ce6c"} Sep 10 22:46:40.122510 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:40.122501 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf" Sep 10 22:46:40.122997 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:40.122542 2577 scope.go:117] "RemoveContainer" containerID="033ae96a371adec2fd49c97b5a5ab43555d79594bfbbf4ddde01207bdcaafed4" Sep 10 22:46:40.146293 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:40.146268 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf"] Sep 10 22:46:40.150669 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:40.150645 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-16d0fd-openshift-default-7f55dbb45b-lxwpf"] Sep 10 22:46:41.750168 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:41.750138 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ddab46d-eebf-43d4-937f-f29835588772" path="/var/lib/kubelet/pods/0ddab46d-eebf-43d4-937f-f29835588772/volumes" Sep 10 22:46:44.854807 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:44.854773 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm"] Sep 10 22:46:44.855297 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:44.855021 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" podUID="074e850d-3c45-49f7-9be1-d56fd2546be8" containerName="maas-api" containerID="cri-o://e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b" gracePeriod=30 Sep 10 22:46:45.121017 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.120955 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:46:45.155407 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.155374 2577 generic.go:356] "Generic (PLEG): container finished" podID="074e850d-3c45-49f7-9be1-d56fd2546be8" containerID="e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b" exitCode=0 Sep 10 22:46:45.155559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.155417 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" Sep 10 22:46:45.155559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.155458 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" event={"ID":"074e850d-3c45-49f7-9be1-d56fd2546be8","Type":"ContainerDied","Data":"e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b"} Sep 10 22:46:45.155559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.155507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm" event={"ID":"074e850d-3c45-49f7-9be1-d56fd2546be8","Type":"ContainerDied","Data":"0dbcdea628f1c7f92ef9df3de814ca6fed9a683918780465239741bddc7e4ced"} Sep 10 22:46:45.155559 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.155531 2577 scope.go:117] "RemoveContainer" containerID="e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b" Sep 10 22:46:45.164772 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.164754 2577 scope.go:117] "RemoveContainer" containerID="e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b" Sep 10 22:46:45.165114 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:46:45.165088 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b\": container with ID starting with e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b not found: ID does not exist" containerID="e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b" Sep 10 22:46:45.165192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.165121 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b"} err="failed to get container status \"e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b\": rpc error: code = NotFound desc = could not find container \"e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b\": container with ID starting with e9a154c3a5a400977cccd2a6a34b9b3e1097ace7ca02fa22612f0b6d026e8c8b not found: ID does not exist" Sep 10 22:46:45.211468 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.211441 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls\") pod \"074e850d-3c45-49f7-9be1-d56fd2546be8\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " Sep 10 22:46:45.211589 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.211481 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w54k6\" (UniqueName: \"kubernetes.io/projected/074e850d-3c45-49f7-9be1-d56fd2546be8-kube-api-access-w54k6\") pod \"074e850d-3c45-49f7-9be1-d56fd2546be8\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " Sep 10 22:46:45.211589 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.211552 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-metrics-certs\") pod \"074e850d-3c45-49f7-9be1-d56fd2546be8\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " Sep 10 22:46:45.211711 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.211681 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074e850d-3c45-49f7-9be1-d56fd2546be8-openshift-service-ca\") pod \"074e850d-3c45-49f7-9be1-d56fd2546be8\" (UID: \"074e850d-3c45-49f7-9be1-d56fd2546be8\") " Sep 10 22:46:45.212128 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.212096 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/074e850d-3c45-49f7-9be1-d56fd2546be8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "074e850d-3c45-49f7-9be1-d56fd2546be8" (UID: "074e850d-3c45-49f7-9be1-d56fd2546be8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:46:45.213763 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.213733 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/074e850d-3c45-49f7-9be1-d56fd2546be8-kube-api-access-w54k6" (OuterVolumeSpecName: "kube-api-access-w54k6") pod "074e850d-3c45-49f7-9be1-d56fd2546be8" (UID: "074e850d-3c45-49f7-9be1-d56fd2546be8"). InnerVolumeSpecName "kube-api-access-w54k6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:46:45.214165 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.214141 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "074e850d-3c45-49f7-9be1-d56fd2546be8" (UID: "074e850d-3c45-49f7-9be1-d56fd2546be8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:46:45.214225 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.214158 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "074e850d-3c45-49f7-9be1-d56fd2546be8" (UID: "074e850d-3c45-49f7-9be1-d56fd2546be8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:46:45.313156 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.313120 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:45.313156 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.313150 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/074e850d-3c45-49f7-9be1-d56fd2546be8-openshift-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:45.313156 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.313160 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/074e850d-3c45-49f7-9be1-d56fd2546be8-maas-api-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:45.313384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.313169 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w54k6\" (UniqueName: \"kubernetes.io/projected/074e850d-3c45-49f7-9be1-d56fd2546be8-kube-api-access-w54k6\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:45.479363 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.479329 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm"] Sep 10 22:46:45.483059 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.483027 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-d28770-55c8c5b48f-dvzjm"] Sep 10 22:46:45.750182 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:45.750099 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="074e850d-3c45-49f7-9be1-d56fd2546be8" path="/var/lib/kubelet/pods/074e850d-3c45-49f7-9be1-d56fd2546be8/volumes" Sep 10 22:46:51.731491 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:51.731454 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s"] Sep 10 22:46:51.731920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:51.731679 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" containerID="cri-o://452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916" gracePeriod=30 Sep 10 22:46:56.991492 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:56.991466 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:46:57.124120 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124089 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-podinfo\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124155 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb8d4\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-kube-api-access-mb8d4\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124176 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-data\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124200 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-envoy\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124224 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istiod-ca-cert\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124283 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124268 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-credential-socket\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124549 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124295 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-certs\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124549 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124324 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-socket\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124549 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124385 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-token\") pod \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\" (UID: \"c12cb822-61a6-44e7-b0aa-404fac00dc0d\") " Sep 10 22:46:57.124708 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124517 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:57.124708 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124580 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:57.124708 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124585 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:46:57.124708 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124682 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.124923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124702 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.124923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124736 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:57.124923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124747 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.124923 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.124770 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:57.126508 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.126484 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-kube-api-access-mb8d4" (OuterVolumeSpecName: "kube-api-access-mb8d4") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "kube-api-access-mb8d4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:46:57.126631 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.126611 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:46:57.126885 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.126861 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:46:57.126963 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.126861 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c12cb822-61a6-44e7-b0aa-404fac00dc0d" (UID: "c12cb822-61a6-44e7-b0aa-404fac00dc0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:46:57.225987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.225967 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.225987 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.225986 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.226133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.225995 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.226133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.226006 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.226133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.226014 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb8d4\" (UniqueName: \"kubernetes.io/projected/c12cb822-61a6-44e7-b0aa-404fac00dc0d-kube-api-access-mb8d4\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.226133 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.226022 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c12cb822-61a6-44e7-b0aa-404fac00dc0d-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:46:57.230607 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.230584 2577 generic.go:356] "Generic (PLEG): container finished" podID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerID="452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916" exitCode=0 Sep 10 22:46:57.230737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.230638 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" Sep 10 22:46:57.230737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.230654 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" event={"ID":"c12cb822-61a6-44e7-b0aa-404fac00dc0d","Type":"ContainerDied","Data":"452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916"} Sep 10 22:46:57.230737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.230698 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s" event={"ID":"c12cb822-61a6-44e7-b0aa-404fac00dc0d","Type":"ContainerDied","Data":"73dc7c7be49f0ee091a477a0365ee696c4837a062f4171beca0d14745be8ec52"} Sep 10 22:46:57.230737 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.230726 2577 scope.go:117] "RemoveContainer" containerID="452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916" Sep 10 22:46:57.241295 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.241278 2577 scope.go:117] "RemoveContainer" containerID="18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085" Sep 10 22:46:57.250522 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.250507 2577 scope.go:117] "RemoveContainer" containerID="452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916" Sep 10 22:46:57.250851 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:46:57.250810 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916\": container with ID starting with 452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916 not found: ID does not exist" containerID="452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916" Sep 10 22:46:57.250950 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.250860 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916"} err="failed to get container status \"452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916\": rpc error: code = NotFound desc = could not find container \"452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916\": container with ID starting with 452b95caef6955ecf0dba41fb1233d1b97b1b2958b63c416c940331cb2152916 not found: ID does not exist" Sep 10 22:46:57.250950 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.250883 2577 scope.go:117] "RemoveContainer" containerID="18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085" Sep 10 22:46:57.251172 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:46:57.251156 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085\": container with ID starting with 18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085 not found: ID does not exist" containerID="18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085" Sep 10 22:46:57.251229 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.251175 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085"} err="failed to get container status \"18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085\": rpc error: code = NotFound desc = could not find container \"18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085\": container with ID starting with 18ec132f09589c981b25762647b74a66ee1832e3080e5f80185048d17b516085 not found: ID does not exist" Sep 10 22:46:57.253473 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.253452 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s"] Sep 10 22:46:57.256682 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.256663 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d28770-openshift-default-66985896b4-fk29s"] Sep 10 22:46:57.750051 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:46:57.750022 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" path="/var/lib/kubelet/pods/c12cb822-61a6-44e7-b0aa-404fac00dc0d/volumes" Sep 10 22:47:09.444043 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:09.444010 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt"] Sep 10 22:47:09.444650 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:09.444274 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" containerID="cri-o://151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6" gracePeriod=30 Sep 10 22:47:13.237811 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:13.237774 2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 22:47:13.238224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:13.237857 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 22:47:14.697855 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.697810 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:47:14.791388 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791348 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-envoy\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791454 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-certs\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791493 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-data\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791532 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-socket\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791735 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791556 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-podinfo\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791735 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791630 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-credential-socket\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791735 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791672 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqjdm\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-kube-api-access-qqjdm\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791735 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791705 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istiod-ca-cert\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.791978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791788 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:47:14.791978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791807 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-data" (OuterVolumeSpecName: "istio-data") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:47:14.792085 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.792064 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:47:14.792135 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.792122 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:47:14.792205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.792152 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.792205 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.791961 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:47:14.792307 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.792170 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-workload-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.792307 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.792232 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-data\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.794093 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.794069 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-kube-api-access-qqjdm" (OuterVolumeSpecName: "kube-api-access-qqjdm") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "kube-api-access-qqjdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:47:14.794207 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.794153 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:47:14.794331 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.794305 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 22:47:14.892580 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.892538 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-token\") pod \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\" (UID: \"3eded38c-eb50-406d-8f02-8cdf3c4e83c0\") " Sep 10 22:47:14.892834 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.892798 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-podinfo\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.892896 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.892841 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-credential-socket\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.892896 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.892857 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqjdm\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-kube-api-access-qqjdm\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.892896 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.892870 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istiod-ca-cert\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.892896 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.892882 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-envoy\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:14.894632 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.894609 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "3eded38c-eb50-406d-8f02-8cdf3c4e83c0" (UID: "3eded38c-eb50-406d-8f02-8cdf3c4e83c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:47:14.993571 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:14.993547 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3eded38c-eb50-406d-8f02-8cdf3c4e83c0-istio-token\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:15.342788 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.342759 2577 generic.go:356] "Generic (PLEG): container finished" podID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerID="151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6" exitCode=0 Sep 10 22:47:15.342969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.342798 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" event={"ID":"3eded38c-eb50-406d-8f02-8cdf3c4e83c0","Type":"ContainerDied","Data":"151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6"} Sep 10 22:47:15.342969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.342807 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" Sep 10 22:47:15.342969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.342852 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt" event={"ID":"3eded38c-eb50-406d-8f02-8cdf3c4e83c0","Type":"ContainerDied","Data":"64b868e42a2438085270c70739a1851b6bcddc9accd96d0bbf60710327617627"} Sep 10 22:47:15.342969 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.342875 2577 scope.go:117] "RemoveContainer" containerID="151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6" Sep 10 22:47:15.352907 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.352797 2577 scope.go:117] "RemoveContainer" containerID="b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4" Sep 10 22:47:15.364267 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.364223 2577 scope.go:117] "RemoveContainer" containerID="151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6" Sep 10 22:47:15.364900 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:47:15.364872 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6\": container with ID starting with 151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6 not found: ID does not exist" containerID="151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6" Sep 10 22:47:15.365028 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.364908 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6"} err="failed to get container status \"151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6\": rpc error: code = NotFound desc = could not find container \"151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6\": container with ID starting with 151abfe87ff8b58434ddbe9634b8f0452053ae740fec5fce854e762b7351aac6 not found: ID does not exist" Sep 10 22:47:15.365028 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.364932 2577 scope.go:117] "RemoveContainer" containerID="b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4" Sep 10 22:47:15.365305 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:47:15.365277 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4\": container with ID starting with b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4 not found: ID does not exist" containerID="b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4" Sep 10 22:47:15.365383 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.365317 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4"} err="failed to get container status \"b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4\": rpc error: code = NotFound desc = could not find container \"b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4\": container with ID starting with b804319568183007e8d2c4c18a56b59007b37f3fa41c25e542462a63350555a4 not found: ID does not exist" Sep 10 22:47:15.366223 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.366207 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt"] Sep 10 22:47:15.368563 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.368541 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-58cd39-openshift-default-59958f88d4-q4llt"] Sep 10 22:47:15.749666 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:15.749589 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" path="/var/lib/kubelet/pods/3eded38c-eb50-406d-8f02-8cdf3c4e83c0/volumes" Sep 10 22:47:20.233026 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.232989 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m"] Sep 10 22:47:20.233554 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.233427 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" podUID="203be1d6-715b-4694-a338-f19ef1c53ea5" containerName="maas-api" containerID="cri-o://7b8b89be83c3a82c09642547a11c27e6a4b37f017bcb9e2c47759c8d3efc872f" gracePeriod=30 Sep 10 22:47:20.264070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.263979 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p"] Sep 10 22:47:20.266484 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.266454 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-33b4cf-29818005-mk87p"] Sep 10 22:47:20.376641 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.376606 2577 generic.go:356] "Generic (PLEG): container finished" podID="203be1d6-715b-4694-a338-f19ef1c53ea5" containerID="7b8b89be83c3a82c09642547a11c27e6a4b37f017bcb9e2c47759c8d3efc872f" exitCode=0 Sep 10 22:47:20.376803 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.376665 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" event={"ID":"203be1d6-715b-4694-a338-f19ef1c53ea5","Type":"ContainerDied","Data":"7b8b89be83c3a82c09642547a11c27e6a4b37f017bcb9e2c47759c8d3efc872f"} Sep 10 22:47:20.499024 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.498959 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:47:20.539766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.539735 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj4zq\" (UniqueName: \"kubernetes.io/projected/203be1d6-715b-4694-a338-f19ef1c53ea5-kube-api-access-hj4zq\") pod \"203be1d6-715b-4694-a338-f19ef1c53ea5\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " Sep 10 22:47:20.539961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.539845 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-maas-api-tls\") pod \"203be1d6-715b-4694-a338-f19ef1c53ea5\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " Sep 10 22:47:20.539961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.539888 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203be1d6-715b-4694-a338-f19ef1c53ea5-openshift-service-ca\") pod \"203be1d6-715b-4694-a338-f19ef1c53ea5\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " Sep 10 22:47:20.539961 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.539921 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-metrics-certs\") pod \"203be1d6-715b-4694-a338-f19ef1c53ea5\" (UID: \"203be1d6-715b-4694-a338-f19ef1c53ea5\") " Sep 10 22:47:20.540221 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.540190 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/203be1d6-715b-4694-a338-f19ef1c53ea5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "203be1d6-715b-4694-a338-f19ef1c53ea5" (UID: "203be1d6-715b-4694-a338-f19ef1c53ea5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:47:20.542249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.542209 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "203be1d6-715b-4694-a338-f19ef1c53ea5" (UID: "203be1d6-715b-4694-a338-f19ef1c53ea5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:47:20.542465 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.542435 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "203be1d6-715b-4694-a338-f19ef1c53ea5" (UID: "203be1d6-715b-4694-a338-f19ef1c53ea5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:47:20.543157 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.543137 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/203be1d6-715b-4694-a338-f19ef1c53ea5-kube-api-access-hj4zq" (OuterVolumeSpecName: "kube-api-access-hj4zq") pod "203be1d6-715b-4694-a338-f19ef1c53ea5" (UID: "203be1d6-715b-4694-a338-f19ef1c53ea5"). InnerVolumeSpecName "kube-api-access-hj4zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:47:20.641192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.641161 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-maas-api-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:20.641192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.641190 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203be1d6-715b-4694-a338-f19ef1c53ea5-openshift-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:20.641192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.641200 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203be1d6-715b-4694-a338-f19ef1c53ea5-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:20.641408 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:20.641208 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj4zq\" (UniqueName: \"kubernetes.io/projected/203be1d6-715b-4694-a338-f19ef1c53ea5-kube-api-access-hj4zq\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:47:21.384431 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:21.384403 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" Sep 10 22:47:21.384845 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:21.384443 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m" event={"ID":"203be1d6-715b-4694-a338-f19ef1c53ea5","Type":"ContainerDied","Data":"c37fc3e35e32cabceff7ccffb4d9c07316e904cd6ed28e1b312c136197b6f885"} Sep 10 22:47:21.384845 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:21.384506 2577 scope.go:117] "RemoveContainer" containerID="7b8b89be83c3a82c09642547a11c27e6a4b37f017bcb9e2c47759c8d3efc872f" Sep 10 22:47:21.411015 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:21.410989 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m"] Sep 10 22:47:21.413520 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:21.413496 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-33b4cf-64968bbc49-hbn2m"] Sep 10 22:47:21.750701 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:21.750624 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="203be1d6-715b-4694-a338-f19ef1c53ea5" path="/var/lib/kubelet/pods/203be1d6-715b-4694-a338-f19ef1c53ea5/volumes" Sep 10 22:47:21.750978 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:47:21.750963 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26c83503-367e-43c6-82b1-32ef9596d563" path="/var/lib/kubelet/pods/26c83503-367e-43c6-82b1-32ef9596d563/volumes" Sep 10 22:48:13.840502 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:48:13.840472 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:48:13.847619 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:48:13.847596 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:50:16.703783 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:16.703748 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h"] Sep 10 22:50:16.704366 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:16.704013 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" podUID="11358f6d-0980-4257-b4ff-e0c254bdedde" containerName="manager" containerID="cri-o://0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6" gracePeriod=10 Sep 10 22:50:16.966275 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:16.966212 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:50:17.073945 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.073899 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt6zw\" (UniqueName: \"kubernetes.io/projected/11358f6d-0980-4257-b4ff-e0c254bdedde-kube-api-access-dt6zw\") pod \"11358f6d-0980-4257-b4ff-e0c254bdedde\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " Sep 10 22:50:17.073945 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.073955 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11358f6d-0980-4257-b4ff-e0c254bdedde-extensions-socket-volume\") pod \"11358f6d-0980-4257-b4ff-e0c254bdedde\" (UID: \"11358f6d-0980-4257-b4ff-e0c254bdedde\") " Sep 10 22:50:17.074396 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.074362 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11358f6d-0980-4257-b4ff-e0c254bdedde-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "11358f6d-0980-4257-b4ff-e0c254bdedde" (UID: "11358f6d-0980-4257-b4ff-e0c254bdedde"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 22:50:17.076098 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.076074 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11358f6d-0980-4257-b4ff-e0c254bdedde-kube-api-access-dt6zw" (OuterVolumeSpecName: "kube-api-access-dt6zw") pod "11358f6d-0980-4257-b4ff-e0c254bdedde" (UID: "11358f6d-0980-4257-b4ff-e0c254bdedde"). InnerVolumeSpecName "kube-api-access-dt6zw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:50:17.174561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.174528 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt6zw\" (UniqueName: \"kubernetes.io/projected/11358f6d-0980-4257-b4ff-e0c254bdedde-kube-api-access-dt6zw\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:50:17.174561 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.174553 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11358f6d-0980-4257-b4ff-e0c254bdedde-extensions-socket-volume\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:50:17.403462 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.403425 2577 generic.go:356] "Generic (PLEG): container finished" podID="11358f6d-0980-4257-b4ff-e0c254bdedde" containerID="0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6" exitCode=0 Sep 10 22:50:17.403666 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.403476 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" Sep 10 22:50:17.403666 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.403509 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" event={"ID":"11358f6d-0980-4257-b4ff-e0c254bdedde","Type":"ContainerDied","Data":"0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6"} Sep 10 22:50:17.403666 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.403565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h" event={"ID":"11358f6d-0980-4257-b4ff-e0c254bdedde","Type":"ContainerDied","Data":"328889d474819133acdefcb36672963695263a3fb4374c98755202ccea649a4b"} Sep 10 22:50:17.403666 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.403587 2577 scope.go:117] "RemoveContainer" containerID="0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6" Sep 10 22:50:17.414844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.414807 2577 scope.go:117] "RemoveContainer" containerID="0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6" Sep 10 22:50:17.415137 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:50:17.415116 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6\": container with ID starting with 0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6 not found: ID does not exist" containerID="0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6" Sep 10 22:50:17.415238 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.415143 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6"} err="failed to get container status \"0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6\": rpc error: code = NotFound desc = could not find container \"0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6\": container with ID starting with 0884ccd3483885f748e533f41661a82288339c3e173546764cb2e341514522c6 not found: ID does not exist" Sep 10 22:50:17.427303 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.427274 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h"] Sep 10 22:50:17.430450 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.430424 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7tt5h"] Sep 10 22:50:17.749063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:50:17.748974 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11358f6d-0980-4257-b4ff-e0c254bdedde" path="/var/lib/kubelet/pods/11358f6d-0980-4257-b4ff-e0c254bdedde/volumes" Sep 10 22:51:14.056487 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:14.056415 2577 scope.go:117] "RemoveContainer" containerID="41f0d8be0224d7f8866f8f282261cc4a2a22bbd99c145b29bf9f7d66bf91e2e4" Sep 10 22:51:14.065381 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:14.065361 2577 scope.go:117] "RemoveContainer" containerID="339aa8f3e8410778384347f2f13a8755357084c13aa4382a3aafec4e45f10986" Sep 10 22:51:17.386789 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.386755 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p"] Sep 10 22:51:17.387230 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387214 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" Sep 10 22:51:17.387278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387232 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" Sep 10 22:51:17.387278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387242 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" Sep 10 22:51:17.387278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387248 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" Sep 10 22:51:17.387278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387256 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" Sep 10 22:51:17.387278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387262 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" Sep 10 22:51:17.387278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387274 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" Sep 10 22:51:17.387278 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387279 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387287 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbcf03be-708f-4375-a31e-9de06f6a9372" containerName="cleanup" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387292 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbcf03be-708f-4375-a31e-9de06f6a9372" containerName="cleanup" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387307 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="074e850d-3c45-49f7-9be1-d56fd2546be8" containerName="maas-api" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387312 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="074e850d-3c45-49f7-9be1-d56fd2546be8" containerName="maas-api" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387320 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11358f6d-0980-4257-b4ff-e0c254bdedde" containerName="manager" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387325 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="11358f6d-0980-4257-b4ff-e0c254bdedde" containerName="manager" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387333 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387338 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387345 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387350 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387356 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203be1d6-715b-4694-a338-f19ef1c53ea5" containerName="maas-api" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387362 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="203be1d6-715b-4694-a338-f19ef1c53ea5" containerName="maas-api" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387373 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26c83503-367e-43c6-82b1-32ef9596d563" containerName="cleanup" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387379 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="26c83503-367e-43c6-82b1-32ef9596d563" containerName="cleanup" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387439 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="074e850d-3c45-49f7-9be1-d56fd2546be8" containerName="maas-api" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387447 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387460 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbcf03be-708f-4375-a31e-9de06f6a9372" containerName="cleanup" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387466 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="203be1d6-715b-4694-a338-f19ef1c53ea5" containerName="maas-api" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387472 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="11358f6d-0980-4257-b4ff-e0c254bdedde" containerName="manager" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387477 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387482 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12cb822-61a6-44e7-b0aa-404fac00dc0d" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387490 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387495 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eded38c-eb50-406d-8f02-8cdf3c4e83c0" containerName="istio-proxy" Sep 10 22:51:17.387494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.387502 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="26c83503-367e-43c6-82b1-32ef9596d563" containerName="cleanup" Sep 10 22:51:17.390637 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.390621 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.398084 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.398061 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qplrh\"" Sep 10 22:51:17.409106 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.409083 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p"] Sep 10 22:51:17.479865 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.479806 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8plrs\" (UniqueName: \"kubernetes.io/projected/3e7f8d15-5a5b-4c4b-bce1-30011bbca435-kube-api-access-8plrs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6xx7p\" (UID: \"3e7f8d15-5a5b-4c4b-bce1-30011bbca435\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.480012 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.479914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e7f8d15-5a5b-4c4b-bce1-30011bbca435-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6xx7p\" (UID: \"3e7f8d15-5a5b-4c4b-bce1-30011bbca435\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.580764 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.580736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8plrs\" (UniqueName: \"kubernetes.io/projected/3e7f8d15-5a5b-4c4b-bce1-30011bbca435-kube-api-access-8plrs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6xx7p\" (UID: \"3e7f8d15-5a5b-4c4b-bce1-30011bbca435\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.580905 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.580781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e7f8d15-5a5b-4c4b-bce1-30011bbca435-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6xx7p\" (UID: \"3e7f8d15-5a5b-4c4b-bce1-30011bbca435\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.581124 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.581107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e7f8d15-5a5b-4c4b-bce1-30011bbca435-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6xx7p\" (UID: \"3e7f8d15-5a5b-4c4b-bce1-30011bbca435\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.590458 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.590433 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8plrs\" (UniqueName: \"kubernetes.io/projected/3e7f8d15-5a5b-4c4b-bce1-30011bbca435-kube-api-access-8plrs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-6xx7p\" (UID: \"3e7f8d15-5a5b-4c4b-bce1-30011bbca435\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.701162 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.701082 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:17.980958 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:17.980933 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p"] Sep 10 22:51:17.983065 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:51:17.983040 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7f8d15_5a5b_4c4b_bce1_30011bbca435.slice/crio-fb6e287e87ec014965c66ffe607f6b32a3128a1acb8b77dafe1610def60c4175 WatchSource:0}: Error finding container fb6e287e87ec014965c66ffe607f6b32a3128a1acb8b77dafe1610def60c4175: Status 404 returned error can't find the container with id fb6e287e87ec014965c66ffe607f6b32a3128a1acb8b77dafe1610def60c4175 Sep 10 22:51:18.753634 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:18.753591 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" event={"ID":"3e7f8d15-5a5b-4c4b-bce1-30011bbca435","Type":"ContainerStarted","Data":"91c8360eec77da0e1e05ecd463262d1774cfc26134f13c9ba191165563f41165"} Sep 10 22:51:18.753634 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:18.753637 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" event={"ID":"3e7f8d15-5a5b-4c4b-bce1-30011bbca435","Type":"ContainerStarted","Data":"fb6e287e87ec014965c66ffe607f6b32a3128a1acb8b77dafe1610def60c4175"} Sep 10 22:51:18.754132 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:18.753732 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:51:18.775104 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:18.775060 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" podStartSLOduration=1.775047273 podStartE2EDuration="1.775047273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:51:18.771193279 +0000 UTC m=+1985.567408756" watchObservedRunningTime="2026-09-10 22:51:18.775047273 +0000 UTC m=+1985.571262749" Sep 10 22:51:29.761676 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:51:29.761647 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-6xx7p" Sep 10 22:52:44.080095 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.080026 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx"] Sep 10 22:52:44.080560 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.080546 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ddab46d-eebf-43d4-937f-f29835588772" containerName="istio-proxy" Sep 10 22:52:44.083709 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.083692 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.086756 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.086735 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ljsfd\"" Sep 10 22:52:44.086900 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.086735 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 22:52:44.086900 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.086736 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 22:52:44.094956 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.094936 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx"] Sep 10 22:52:44.131048 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.131018 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66dbx\" (UniqueName: \"kubernetes.io/projected/0799c679-1fd4-4d87-b14f-72691190c5bc-kube-api-access-66dbx\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.131162 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.131054 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0799c679-1fd4-4d87-b14f-72691190c5bc-openshift-service-ca\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.131162 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.131086 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-metrics-certs\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.131237 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.131209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-maas-api-tls\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.232137 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.232101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-maas-api-tls\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.232315 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.232151 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66dbx\" (UniqueName: \"kubernetes.io/projected/0799c679-1fd4-4d87-b14f-72691190c5bc-kube-api-access-66dbx\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.232315 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.232256 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0799c679-1fd4-4d87-b14f-72691190c5bc-openshift-service-ca\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.232315 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.232310 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-metrics-certs\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.232932 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.232912 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0799c679-1fd4-4d87-b14f-72691190c5bc-openshift-service-ca\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.234742 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.234710 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-metrics-certs\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.234858 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.234745 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-maas-api-tls\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.243032 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.243009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66dbx\" (UniqueName: \"kubernetes.io/projected/0799c679-1fd4-4d87-b14f-72691190c5bc-kube-api-access-66dbx\") pod \"maas-api-7745fd994b-8pjvx\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.395173 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.395091 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:44.672339 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.672313 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx"] Sep 10 22:52:44.674671 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:52:44.674643 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0799c679_1fd4_4d87_b14f_72691190c5bc.slice/crio-4771d957fc4ad32e60ef6d4290cdd6eb2ab5894255592403bff01fce5e4c0b33 WatchSource:0}: Error finding container 4771d957fc4ad32e60ef6d4290cdd6eb2ab5894255592403bff01fce5e4c0b33: Status 404 returned error can't find the container with id 4771d957fc4ad32e60ef6d4290cdd6eb2ab5894255592403bff01fce5e4c0b33 Sep 10 22:52:44.676450 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:44.676431 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 22:52:45.262388 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:45.262357 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" event={"ID":"0799c679-1fd4-4d87-b14f-72691190c5bc","Type":"ContainerStarted","Data":"42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526"} Sep 10 22:52:45.262687 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:45.262417 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:45.262687 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:45.262436 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" event={"ID":"0799c679-1fd4-4d87-b14f-72691190c5bc","Type":"ContainerStarted","Data":"4771d957fc4ad32e60ef6d4290cdd6eb2ab5894255592403bff01fce5e4c0b33"} Sep 10 22:52:45.281303 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:45.281247 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" podStartSLOduration=0.762687896 podStartE2EDuration="1.281229833s" totalImagesPullingTime="518.541937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:52:45.277756725 +0000 UTC m=+2072.073972184" watchObservedRunningTime="2026-09-10 22:52:45.281229833 +0000 UTC m=+2072.077445314" Sep 10 22:52:51.274934 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.274905 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:51.514219 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.514174 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-867d7774cd-drmbk"] Sep 10 22:52:51.517950 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.517918 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.526682 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.526659 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-867d7774cd-drmbk"] Sep 10 22:52:51.596810 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.596775 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42703dd6-d8d5-477d-a73a-efcc8d95db52-maas-api-tls\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.596963 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.596848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fgw4\" (UniqueName: \"kubernetes.io/projected/42703dd6-d8d5-477d-a73a-efcc8d95db52-kube-api-access-5fgw4\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.597011 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.596978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42703dd6-d8d5-477d-a73a-efcc8d95db52-openshift-service-ca\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.597049 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.597015 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42703dd6-d8d5-477d-a73a-efcc8d95db52-metrics-certs\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.697721 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.697692 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42703dd6-d8d5-477d-a73a-efcc8d95db52-metrics-certs\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.697865 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.697741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42703dd6-d8d5-477d-a73a-efcc8d95db52-maas-api-tls\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.697918 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.697863 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fgw4\" (UniqueName: \"kubernetes.io/projected/42703dd6-d8d5-477d-a73a-efcc8d95db52-kube-api-access-5fgw4\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.698020 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.698004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42703dd6-d8d5-477d-a73a-efcc8d95db52-openshift-service-ca\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.698707 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.698676 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42703dd6-d8d5-477d-a73a-efcc8d95db52-openshift-service-ca\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.700299 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.700281 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42703dd6-d8d5-477d-a73a-efcc8d95db52-metrics-certs\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.700390 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.700319 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42703dd6-d8d5-477d-a73a-efcc8d95db52-maas-api-tls\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.707558 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.707538 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fgw4\" (UniqueName: \"kubernetes.io/projected/42703dd6-d8d5-477d-a73a-efcc8d95db52-kube-api-access-5fgw4\") pod \"maas-api-867d7774cd-drmbk\" (UID: \"42703dd6-d8d5-477d-a73a-efcc8d95db52\") " pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:51.829989 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:51.829913 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:52.104872 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:52.104846 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-867d7774cd-drmbk"] Sep 10 22:52:52.106593 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:52:52.106559 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42703dd6_d8d5_477d_a73a_efcc8d95db52.slice/crio-a6ecc89eac6dde57547a9c74cf3972990514bedf2829396c7bf8408069f4da5d WatchSource:0}: Error finding container a6ecc89eac6dde57547a9c74cf3972990514bedf2829396c7bf8408069f4da5d: Status 404 returned error can't find the container with id a6ecc89eac6dde57547a9c74cf3972990514bedf2829396c7bf8408069f4da5d Sep 10 22:52:52.304275 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:52.304240 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" event={"ID":"42703dd6-d8d5-477d-a73a-efcc8d95db52","Type":"ContainerStarted","Data":"a6ecc89eac6dde57547a9c74cf3972990514bedf2829396c7bf8408069f4da5d"} Sep 10 22:52:53.311844 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:53.311784 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" event={"ID":"42703dd6-d8d5-477d-a73a-efcc8d95db52","Type":"ContainerStarted","Data":"06ba8581b22a1ebf312cb0f2be262f5fe36c1e9b1cfc3dd4c250bb89c44b58e2"} Sep 10 22:52:53.312238 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:53.311867 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:53.328790 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:53.328744 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" podStartSLOduration=2.045505601 podStartE2EDuration="2.328729445s" totalImagesPullingTime="283.223844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:52:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:52:53.327025047 +0000 UTC m=+2080.123240525" watchObservedRunningTime="2026-09-10 22:52:53.328729445 +0000 UTC m=+2080.124944945" Sep 10 22:52:59.327033 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.327001 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-867d7774cd-drmbk" Sep 10 22:52:59.426323 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.426290 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx"] Sep 10 22:52:59.426526 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.426481 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" podUID="0799c679-1fd4-4d87-b14f-72691190c5bc" containerName="maas-api" containerID="cri-o://42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526" gracePeriod=30 Sep 10 22:52:59.675196 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.675171 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:52:59.774152 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.774120 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66dbx\" (UniqueName: \"kubernetes.io/projected/0799c679-1fd4-4d87-b14f-72691190c5bc-kube-api-access-66dbx\") pod \"0799c679-1fd4-4d87-b14f-72691190c5bc\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " Sep 10 22:52:59.774311 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.774182 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-maas-api-tls\") pod \"0799c679-1fd4-4d87-b14f-72691190c5bc\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " Sep 10 22:52:59.774311 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.774223 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0799c679-1fd4-4d87-b14f-72691190c5bc-openshift-service-ca\") pod \"0799c679-1fd4-4d87-b14f-72691190c5bc\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " Sep 10 22:52:59.774311 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.774275 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-metrics-certs\") pod \"0799c679-1fd4-4d87-b14f-72691190c5bc\" (UID: \"0799c679-1fd4-4d87-b14f-72691190c5bc\") " Sep 10 22:52:59.774586 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.774557 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0799c679-1fd4-4d87-b14f-72691190c5bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0799c679-1fd4-4d87-b14f-72691190c5bc" (UID: "0799c679-1fd4-4d87-b14f-72691190c5bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 22:52:59.776521 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.776482 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0799c679-1fd4-4d87-b14f-72691190c5bc" (UID: "0799c679-1fd4-4d87-b14f-72691190c5bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:52:59.776648 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.776544 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0799c679-1fd4-4d87-b14f-72691190c5bc-kube-api-access-66dbx" (OuterVolumeSpecName: "kube-api-access-66dbx") pod "0799c679-1fd4-4d87-b14f-72691190c5bc" (UID: "0799c679-1fd4-4d87-b14f-72691190c5bc"). InnerVolumeSpecName "kube-api-access-66dbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 22:52:59.776648 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.776569 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0799c679-1fd4-4d87-b14f-72691190c5bc" (UID: "0799c679-1fd4-4d87-b14f-72691190c5bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 22:52:59.875174 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.875109 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-metrics-certs\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:52:59.875174 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.875134 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66dbx\" (UniqueName: \"kubernetes.io/projected/0799c679-1fd4-4d87-b14f-72691190c5bc-kube-api-access-66dbx\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:52:59.875174 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.875145 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0799c679-1fd4-4d87-b14f-72691190c5bc-maas-api-tls\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:52:59.875174 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:52:59.875153 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0799c679-1fd4-4d87-b14f-72691190c5bc-openshift-service-ca\") on node \"ip-10-0-142-235.ec2.internal\" DevicePath \"\"" Sep 10 22:53:00.356869 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.356803 2577 generic.go:356] "Generic (PLEG): container finished" podID="0799c679-1fd4-4d87-b14f-72691190c5bc" containerID="42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526" exitCode=0 Sep 10 22:53:00.356869 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.356869 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" Sep 10 22:53:00.357325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.356891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" event={"ID":"0799c679-1fd4-4d87-b14f-72691190c5bc","Type":"ContainerDied","Data":"42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526"} Sep 10 22:53:00.357325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.356949 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx" event={"ID":"0799c679-1fd4-4d87-b14f-72691190c5bc","Type":"ContainerDied","Data":"4771d957fc4ad32e60ef6d4290cdd6eb2ab5894255592403bff01fce5e4c0b33"} Sep 10 22:53:00.357325 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.356971 2577 scope.go:117] "RemoveContainer" containerID="42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526" Sep 10 22:53:00.367349 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.367330 2577 scope.go:117] "RemoveContainer" containerID="42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526" Sep 10 22:53:00.367609 ip-10-0-142-235 kubenswrapper[2577]: E0910 22:53:00.367589 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526\": container with ID starting with 42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526 not found: ID does not exist" containerID="42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526" Sep 10 22:53:00.367653 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.367618 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526"} err="failed to get container status \"42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526\": rpc error: code = NotFound desc = could not find container \"42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526\": container with ID starting with 42cc5b988ab70acbed43a71e3359cf0f10ab2d01e5536d092ed6b59d68b42526 not found: ID does not exist" Sep 10 22:53:00.379488 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.379467 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx"] Sep 10 22:53:00.385473 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:00.385452 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7745fd994b-8pjvx"] Sep 10 22:53:01.749772 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:01.749741 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0799c679-1fd4-4d87-b14f-72691190c5bc" path="/var/lib/kubelet/pods/0799c679-1fd4-4d87-b14f-72691190c5bc/volumes" Sep 10 22:53:13.881048 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:13.881014 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:53:13.890946 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:13.890917 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:53:19.111698 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:19.111660 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7cng5_18ac3157-48f5-4ec2-897a-68b581b9e537/manager/0.log" Sep 10 22:53:19.799220 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:19.799190 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c969df88c-vr5ft_ae81311b-cd92-434b-be58-71192a85ab62/copy-manifests/0.log" Sep 10 22:53:19.811389 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:19.811362 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c969df88c-vr5ft_ae81311b-cd92-434b-be58-71192a85ab62/manager/0.log" Sep 10 22:53:20.146759 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:20.146687 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-867d7774cd-drmbk_42703dd6-d8d5-477d-a73a-efcc8d95db52/maas-api/0.log" Sep 10 22:53:20.366653 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:20.366627 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817990-wbkzk_12c445bf-9821-4312-9a2f-e800edfa3263/cleanup/0.log" Sep 10 22:53:20.582063 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:20.582032 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jxfr9_d33497e9-eb38-4d74-9d83-576e7cda48b8/postgres/0.log" Sep 10 22:53:21.781071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:21.781036 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-6xx7p_3e7f8d15-5a5b-4c4b-bce1-30011bbca435/manager/0.log" Sep 10 22:53:22.009246 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:22.009216 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-lq8db_c2453d00-64cd-4eae-a126-2cf3428a04d9/manager/0.log" Sep 10 22:53:22.452551 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:22.452522 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-njkbb_dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2/discovery/0.log" Sep 10 22:53:22.668411 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:22.668383 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54fc6bbcb5-dpr5w_98a750a4-0cc1-4692-b5eb-9686a9c7634b/kube-auth-proxy/0.log" Sep 10 22:53:22.906696 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:22.906669 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jcw7s_43ee3d49-6550-42b3-bafc-c70e46d7f35b/payload-pre-processing/0.log" Sep 10 22:53:23.022204 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:23.022173 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t7dns_471c8294-5aad-47d3-8392-d937af251324/payload-processing/0.log" Sep 10 22:53:23.136625 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:23.136598 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6cf6d56fcd-xqxl8_9dfbb79d-2457-4d91-8bd0-fe718a39e9f4/router/0.log" Sep 10 22:53:23.578192 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:23.578162 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-87t4l_ec33c8f0-a7f8-4fe9-9459-c0b85cf5ddc7/main/0.log" Sep 10 22:53:32.215082 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:32.215045 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g7tpk_d0d3d831-2afd-4647-9c64-22c55eccf863/global-pull-secret-syncer/0.log" Sep 10 22:53:32.387568 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:32.387534 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-scrkj_d2b6e10a-8490-4ef2-9146-601fb419ad66/konnectivity-agent/0.log" Sep 10 22:53:32.457540 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:32.457511 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-235.ec2.internal_be714b5e0febb4494d9a83009ae5a158/haproxy/0.log" Sep 10 22:53:37.162012 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:37.161985 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-6xx7p_3e7f8d15-5a5b-4c4b-bce1-30011bbca435/manager/0.log" Sep 10 22:53:37.248980 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:37.248948 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-lq8db_c2453d00-64cd-4eae-a126-2cf3428a04d9/manager/0.log" Sep 10 22:53:38.652368 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.652346 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02c61abd-a483-4bbd-bdfd-d982c9003a00/alertmanager/0.log" Sep 10 22:53:38.673766 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.673740 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02c61abd-a483-4bbd-bdfd-d982c9003a00/config-reloader/0.log" Sep 10 22:53:38.695127 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.695108 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02c61abd-a483-4bbd-bdfd-d982c9003a00/kube-rbac-proxy-web/0.log" Sep 10 22:53:38.717088 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.717072 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02c61abd-a483-4bbd-bdfd-d982c9003a00/kube-rbac-proxy/0.log" Sep 10 22:53:38.738184 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.738164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02c61abd-a483-4bbd-bdfd-d982c9003a00/kube-rbac-proxy-metric/0.log" Sep 10 22:53:38.758174 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.758157 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02c61abd-a483-4bbd-bdfd-d982c9003a00/prom-label-proxy/0.log" Sep 10 22:53:38.779570 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.779556 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_02c61abd-a483-4bbd-bdfd-d982c9003a00/init-config-reloader/0.log" Sep 10 22:53:38.820684 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.820662 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-4pkxw_d6f97834-3301-409b-a2ea-a6fd959d3068/cluster-monitoring-operator/0.log" Sep 10 22:53:38.929154 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.929081 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-746fddfb69-zcrbl_7a1fed85-cc3f-4595-8798-0c8fac83f1df/metrics-server/0.log" Sep 10 22:53:38.959596 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.959576 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-w652v_e7bc890e-db81-4dc4-877b-bef02e2bf7a1/monitoring-plugin/0.log" Sep 10 22:53:38.990579 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:38.990562 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7js62_4dcfaacc-843c-4e92-9570-bd0fbde357d3/node-exporter/0.log" Sep 10 22:53:39.009598 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.009580 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7js62_4dcfaacc-843c-4e92-9570-bd0fbde357d3/kube-rbac-proxy/0.log" Sep 10 22:53:39.033964 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.033943 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7js62_4dcfaacc-843c-4e92-9570-bd0fbde357d3/init-textfile/0.log" Sep 10 22:53:39.297927 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.297898 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-47cr8_a1ba225a-2b83-4110-be11-59a10739dac6/kube-rbac-proxy-main/0.log" Sep 10 22:53:39.324091 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.324065 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-47cr8_a1ba225a-2b83-4110-be11-59a10739dac6/kube-rbac-proxy-self/0.log" Sep 10 22:53:39.351560 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.351533 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-47cr8_a1ba225a-2b83-4110-be11-59a10739dac6/openshift-state-metrics/0.log" Sep 10 22:53:39.733568 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.733543 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64966f776b-q8fdn_694690b8-b2a4-46c5-bad4-f2c313d3254b/thanos-query/0.log" Sep 10 22:53:39.755249 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.755228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64966f776b-q8fdn_694690b8-b2a4-46c5-bad4-f2c313d3254b/kube-rbac-proxy-web/0.log" Sep 10 22:53:39.776070 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.776051 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64966f776b-q8fdn_694690b8-b2a4-46c5-bad4-f2c313d3254b/kube-rbac-proxy/0.log" Sep 10 22:53:39.796079 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.796060 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64966f776b-q8fdn_694690b8-b2a4-46c5-bad4-f2c313d3254b/prom-label-proxy/0.log" Sep 10 22:53:39.816256 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.816242 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64966f776b-q8fdn_694690b8-b2a4-46c5-bad4-f2c313d3254b/kube-rbac-proxy-rules/0.log" Sep 10 22:53:39.834489 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:39.834465 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64966f776b-q8fdn_694690b8-b2a4-46c5-bad4-f2c313d3254b/kube-rbac-proxy-metrics/0.log" Sep 10 22:53:41.225368 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:41.225338 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-rlb5v_ef9fa3f2-f8a4-40e2-aa74-de5824a3b356/networking-console-plugin/0.log" Sep 10 22:53:41.850909 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:41.850876 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/1.log" Sep 10 22:53:41.855008 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:41.854986 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-866j5_84a7c959-b185-469d-8a42-30171c664a31/console-operator/2.log" Sep 10 22:53:42.255676 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.255593 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h"] Sep 10 22:53:42.256071 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.256056 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0799c679-1fd4-4d87-b14f-72691190c5bc" containerName="maas-api" Sep 10 22:53:42.256118 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.256073 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0799c679-1fd4-4d87-b14f-72691190c5bc" containerName="maas-api" Sep 10 22:53:42.256150 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.256143 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0799c679-1fd4-4d87-b14f-72691190c5bc" containerName="maas-api" Sep 10 22:53:42.259565 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.259549 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.262380 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.262355 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n6fx2\"/\"kube-root-ca.crt\"" Sep 10 22:53:42.263102 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.263086 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n6fx2\"/\"default-dockercfg-bc8jm\"" Sep 10 22:53:42.263193 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.263153 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n6fx2\"/\"openshift-service-ca.crt\"" Sep 10 22:53:42.272951 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.272928 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h"] Sep 10 22:53:42.347243 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.347212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-lib-modules\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.347387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.347252 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-proc\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.347387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.347316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-podres\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.347387 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.347351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-sys\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.347494 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.347388 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44n76\" (UniqueName: \"kubernetes.io/projected/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-kube-api-access-44n76\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.434462 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.434436 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7dd66f7944-j62zz_0261df3f-a1ae-480d-8e50-413d425fa8f2/console/0.log" Sep 10 22:53:42.448232 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448200 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44n76\" (UniqueName: \"kubernetes.io/projected/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-kube-api-access-44n76\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448272 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-lib-modules\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448300 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-proc\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448384 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448381 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-proc\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-podres\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448455 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-lib-modules\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448475 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-sys\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448555 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448520 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-sys\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.448692 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.448564 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-podres\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.457635 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.457606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44n76\" (UniqueName: \"kubernetes.io/projected/c3d1580d-dde5-4f68-b4c4-b10c3f047c1d-kube-api-access-44n76\") pod \"perf-node-gather-daemonset-r965h\" (UID: \"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d\") " pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.569604 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.569573 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:42.859762 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:42.859736 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h"] Sep 10 22:53:42.861168 ip-10-0-142-235 kubenswrapper[2577]: W0910 22:53:42.861133 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc3d1580d_dde5_4f68_b4c4_b10c3f047c1d.slice/crio-0f18937a5a2ea7a738ebeb531e55e73a1d948ca71e5601701f83b804b5da6ec7 WatchSource:0}: Error finding container 0f18937a5a2ea7a738ebeb531e55e73a1d948ca71e5601701f83b804b5da6ec7: Status 404 returned error can't find the container with id 0f18937a5a2ea7a738ebeb531e55e73a1d948ca71e5601701f83b804b5da6ec7 Sep 10 22:53:43.252566 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:43.252481 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-frh8v_75d82860-197d-4819-9059-222e9c024ddf/volume-data-source-validator/0.log" Sep 10 22:53:43.607410 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:43.607367 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" event={"ID":"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d","Type":"ContainerStarted","Data":"1db5a49752ec02d8ad5d60d24a8f6653709483eaa88065a7394b5e7b510a6804"} Sep 10 22:53:43.607800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:43.607431 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" event={"ID":"c3d1580d-dde5-4f68-b4c4-b10c3f047c1d","Type":"ContainerStarted","Data":"0f18937a5a2ea7a738ebeb531e55e73a1d948ca71e5601701f83b804b5da6ec7"} Sep 10 22:53:43.607800 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:43.607469 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:43.628284 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:43.628245 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" podStartSLOduration=1.628231187 podStartE2EDuration="1.628231187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:53:43.626037022 +0000 UTC m=+2130.422252499" watchObservedRunningTime="2026-09-10 22:53:43.628231187 +0000 UTC m=+2130.424446664" Sep 10 22:53:44.571767 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:44.571734 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6lz6l_2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20/dns/0.log" Sep 10 22:53:44.595115 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:44.595094 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6lz6l_2f59c5a1-f8f2-4ee2-96de-ea377a4d7c20/kube-rbac-proxy/0.log" Sep 10 22:53:44.824688 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:44.824603 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nj85g_63da62fc-f1bc-49e9-b2ff-0a39cf81331f/dns-node-resolver/0.log" Sep 10 22:53:45.445879 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:45.445847 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rstb7_57467133-81f3-434e-ae9f-1f07cd84ceba/node-ca/0.log" Sep 10 22:53:46.681149 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:46.681120 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-njkbb_dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2/discovery/0.log" Sep 10 22:53:46.724562 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:46.724537 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54fc6bbcb5-dpr5w_98a750a4-0cc1-4692-b5eb-9686a9c7634b/kube-auth-proxy/0.log" Sep 10 22:53:46.793381 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:46.793351 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jcw7s_43ee3d49-6550-42b3-bafc-c70e46d7f35b/payload-pre-processing/0.log" Sep 10 22:53:46.829247 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:46.829219 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t7dns_471c8294-5aad-47d3-8392-d937af251324/payload-processing/0.log" Sep 10 22:53:46.855875 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:46.855847 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6cf6d56fcd-xqxl8_9dfbb79d-2457-4d91-8bd0-fe718a39e9f4/router/0.log" Sep 10 22:53:47.741382 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:47.741341 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n9pms_ef748efe-3001-4cdf-b168-2b215ee1cd9a/serve-healthcheck-canary/0.log" Sep 10 22:53:48.236114 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:48.236064 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-jv4bp_099a0642-5254-4571-8da9-e6cce52ff27f/insights-operator/0.log" Sep 10 22:53:48.236344 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:48.236326 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-jv4bp_099a0642-5254-4571-8da9-e6cce52ff27f/insights-operator/1.log" Sep 10 22:53:48.260831 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:48.260795 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hd4lz_52cf965f-2db0-45c4-b930-5cb0908b6499/kube-rbac-proxy/0.log" Sep 10 22:53:48.283786 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:48.283760 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hd4lz_52cf965f-2db0-45c4-b930-5cb0908b6499/exporter/0.log" Sep 10 22:53:48.306853 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:48.306808 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hd4lz_52cf965f-2db0-45c4-b930-5cb0908b6499/extractor/0.log" Sep 10 22:53:49.623761 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:49.623716 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n6fx2/perf-node-gather-daemonset-r965h" Sep 10 22:53:50.675664 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:50.675629 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7cng5_18ac3157-48f5-4ec2-897a-68b581b9e537/manager/0.log" Sep 10 22:53:51.001033 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:51.000895 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c969df88c-vr5ft_ae81311b-cd92-434b-be58-71192a85ab62/manager/0.log" Sep 10 22:53:51.018316 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:51.018299 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c969df88c-vr5ft_ae81311b-cd92-434b-be58-71192a85ab62/copy-manifests/0.log" Sep 10 22:53:52.532905 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:52.532874 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-njkbb_dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2/discovery/0.log" Sep 10 22:53:52.560379 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:52.560353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54fc6bbcb5-dpr5w_98a750a4-0cc1-4692-b5eb-9686a9c7634b/kube-auth-proxy/0.log" Sep 10 22:53:52.616515 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:52.616492 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jcw7s_43ee3d49-6550-42b3-bafc-c70e46d7f35b/payload-pre-processing/0.log" Sep 10 22:53:52.644988 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:52.644966 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t7dns_471c8294-5aad-47d3-8392-d937af251324/payload-processing/0.log" Sep 10 22:53:52.657780 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:52.657757 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6cf6d56fcd-xqxl8_9dfbb79d-2457-4d91-8bd0-fe718a39e9f4/router/0.log" Sep 10 22:53:54.275427 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:54.275394 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-njkbb_dfa98b9b-ccfe-423c-9bc1-d3bc780e88d2/discovery/0.log" Sep 10 22:53:54.308131 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:54.308110 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-54fc6bbcb5-dpr5w_98a750a4-0cc1-4692-b5eb-9686a9c7634b/kube-auth-proxy/0.log" Sep 10 22:53:54.364107 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:54.364079 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-jcw7s_43ee3d49-6550-42b3-bafc-c70e46d7f35b/payload-pre-processing/0.log" Sep 10 22:53:54.394787 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:54.394760 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-t7dns_471c8294-5aad-47d3-8392-d937af251324/payload-processing/0.log" Sep 10 22:53:54.411173 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:54.411154 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6cf6d56fcd-xqxl8_9dfbb79d-2457-4d91-8bd0-fe718a39e9f4/router/0.log" Sep 10 22:53:57.700086 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:57.700032 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-ndxsf_13205e70-75c2-4340-a014-0f3d44963cbd/kube-storage-version-migrator-operator/1.log" Sep 10 22:53:57.700920 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:57.700901 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-ndxsf_13205e70-75c2-4340-a014-0f3d44963cbd/kube-storage-version-migrator-operator/0.log" Sep 10 22:53:58.905224 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:58.905198 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j454d_079e30d5-078c-4982-981d-f4df3a17834b/kube-multus-additional-cni-plugins/0.log" Sep 10 22:53:58.924193 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:58.924171 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j454d_079e30d5-078c-4982-981d-f4df3a17834b/egress-router-binary-copy/0.log" Sep 10 22:53:58.942578 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:58.942558 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j454d_079e30d5-078c-4982-981d-f4df3a17834b/cni-plugins/0.log" Sep 10 22:53:58.963583 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:58.963562 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j454d_079e30d5-078c-4982-981d-f4df3a17834b/bond-cni-plugin/0.log" Sep 10 22:53:58.983066 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:58.983041 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j454d_079e30d5-078c-4982-981d-f4df3a17834b/routeoverride-cni/0.log" Sep 10 22:53:59.002410 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:59.002386 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j454d_079e30d5-078c-4982-981d-f4df3a17834b/whereabouts-cni-bincopy/0.log" Sep 10 22:53:59.022135 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:59.022113 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j454d_079e30d5-078c-4982-981d-f4df3a17834b/whereabouts-cni/0.log" Sep 10 22:53:59.210651 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:59.210575 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fcrdb_5308bd19-fb64-46c0-95aa-db0504e62f76/kube-multus/0.log" Sep 10 22:53:59.322656 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:59.322630 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9dbrx_0521c5d3-62eb-4ea7-b2e4-a2da4244d291/network-metrics-daemon/0.log" Sep 10 22:53:59.341144 ip-10-0-142-235 kubenswrapper[2577]: I0910 22:53:59.341117 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9dbrx_0521c5d3-62eb-4ea7-b2e4-a2da4244d291/kube-rbac-proxy/0.log"