Sep 10 14:42:21.870957 ip-10-0-136-202 systemd[1]: Starting Kubernetes Kubelet... Sep 10 14:42:22.223338 ip-10-0-136-202 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 14:42:22.223338 ip-10-0-136-202 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 10 14:42:22.223338 ip-10-0-136-202 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 14:42:22.223338 ip-10-0-136-202 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 10 14:42:22.223338 ip-10-0-136-202 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 10 14:42:22.224997 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.224912 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 10 14:42:22.229454 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229436 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 14:42:22.229454 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229452 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 14:42:22.229454 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229457 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229462 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229466 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229470 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229474 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229478 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229481 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229485 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229490 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229494 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229498 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229506 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229510 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229514 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229518 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229521 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229526 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229530 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229534 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229538 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 14:42:22.229649 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229543 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229547 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229550 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229554 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229559 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229564 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229569 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229573 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229578 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229583 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229587 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229591 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229595 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229599 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229604 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229623 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229628 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229632 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229636 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229641 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 14:42:22.230469 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229645 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229648 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229653 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229658 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229662 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229666 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229674 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229680 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229685 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229690 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229695 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229699 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229705 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229709 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229713 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229717 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229722 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229726 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229734 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 14:42:22.230990 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229741 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229746 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229750 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229757 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229762 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229766 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229771 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229777 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229781 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229785 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229790 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229794 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229799 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229803 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229807 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229811 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229816 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229821 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229826 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 14:42:22.231832 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229830 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229834 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229838 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229842 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229846 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229850 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229854 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229859 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229863 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229867 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229871 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229875 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229879 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229883 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229887 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229891 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229896 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229900 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.229904 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231642 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 14:42:22.232650 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231655 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231660 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231667 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231674 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231682 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231687 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231691 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231696 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231701 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231705 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231709 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231714 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231719 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231723 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231730 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231736 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231742 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231746 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231751 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 14:42:22.233483 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231756 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231761 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231765 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231769 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231774 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231779 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231783 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231787 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231791 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231796 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231800 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231804 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231809 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231813 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231818 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231822 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231826 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231830 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231834 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231838 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 14:42:22.234337 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231842 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231845 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231849 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231853 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231857 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231863 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231867 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231871 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231875 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231879 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231884 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231888 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231892 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231896 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231900 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231905 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231909 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231914 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231918 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 14:42:22.235045 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231923 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231927 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231931 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231938 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231942 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231946 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231950 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231954 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231958 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231962 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231966 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231971 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231975 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231979 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231983 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231987 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231991 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.231996 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232002 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232009 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 14:42:22.235586 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232013 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232018 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232023 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232028 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232032 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232036 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232041 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232046 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232051 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232055 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232060 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232064 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232068 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232072 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232077 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232081 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232085 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232089 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232093 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232097 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 14:42:22.236121 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232715 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232724 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232728 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232733 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232737 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232741 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232745 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232750 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232754 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232758 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232762 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232767 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232772 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232776 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232780 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232787 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232792 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232797 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232801 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232805 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 14:42:22.236788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232810 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232814 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232819 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232824 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232828 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232832 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232837 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232841 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232845 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232849 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232853 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232857 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232862 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232866 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232870 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232874 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232878 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232882 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232886 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232891 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 14:42:22.237309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232895 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232899 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232904 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232910 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232916 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232921 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232924 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232929 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232933 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232937 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232942 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232946 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232953 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232959 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232964 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232970 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232975 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232979 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 14:42:22.237844 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232984 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232988 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232992 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.232996 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233000 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233004 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233008 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233012 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233016 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233020 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233024 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233029 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233033 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233037 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233041 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233045 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233049 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233054 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233059 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233064 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 14:42:22.238378 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233068 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233072 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233076 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233081 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233088 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233092 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233097 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233101 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233106 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233110 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233114 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233118 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233122 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233126 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233131 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233135 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233140 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233144 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233148 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 14:42:22.238914 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233152 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.233156 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233260 2584 flags.go:64] FLAG: --address="0.0.0.0" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233327 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233338 2584 flags.go:64] FLAG: --anonymous-auth="true" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233345 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233351 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233356 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233362 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233369 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233374 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233379 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233384 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233390 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233395 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233400 2584 flags.go:64] FLAG: --cgroup-root="" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233405 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233412 2584 flags.go:64] FLAG: --client-ca-file="" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233417 2584 flags.go:64] FLAG: --cloud-provider="external" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233421 2584 flags.go:64] FLAG: --cluster-dns="[]" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233427 2584 flags.go:64] FLAG: --cluster-domain="" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233434 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233439 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233444 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 10 14:42:22.239374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233450 2584 flags.go:64] FLAG: --container-log-max-files="5" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233456 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233461 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233466 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233471 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233476 2584 flags.go:64] FLAG: --contention-profiling="false" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233480 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233485 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233489 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233494 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233500 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233505 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233510 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233515 2584 flags.go:64] FLAG: --enable-load-reader="false" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233519 2584 flags.go:64] FLAG: --enable-server="true" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233524 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233535 2584 flags.go:64] FLAG: --event-burst="100" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233540 2584 flags.go:64] FLAG: --event-qps="50" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233544 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233549 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233554 2584 flags.go:64] FLAG: --eviction-hard="" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233560 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233566 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233571 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233576 2584 flags.go:64] FLAG: --eviction-soft="" Sep 10 14:42:22.239966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233582 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233587 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233591 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233596 2584 flags.go:64] FLAG: --experimental-mounter-path="" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233603 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233625 2584 flags.go:64] FLAG: --fail-swap-on="true" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233630 2584 flags.go:64] FLAG: --feature-gates="" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233636 2584 flags.go:64] FLAG: --file-check-frequency="20s" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233643 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233647 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233652 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233657 2584 flags.go:64] FLAG: --healthz-port="10248" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233662 2584 flags.go:64] FLAG: --help="false" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233666 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233672 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233676 2584 flags.go:64] FLAG: --http-check-frequency="20s" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233681 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233687 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233692 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233697 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233701 2584 flags.go:64] FLAG: --image-service-endpoint="" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233706 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233710 2584 flags.go:64] FLAG: --kube-api-burst="100" Sep 10 14:42:22.240621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233715 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233720 2584 flags.go:64] FLAG: --kube-api-qps="50" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233725 2584 flags.go:64] FLAG: --kube-reserved="" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233730 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233735 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233740 2584 flags.go:64] FLAG: --kubelet-cgroups="" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233745 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233751 2584 flags.go:64] FLAG: --lock-file="" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233755 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233762 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233767 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233776 2584 flags.go:64] FLAG: --log-json-split-stream="false" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233780 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233786 2584 flags.go:64] FLAG: --log-text-split-stream="false" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233791 2584 flags.go:64] FLAG: --logging-format="text" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233796 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233801 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233806 2584 flags.go:64] FLAG: --manifest-url="" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233810 2584 flags.go:64] FLAG: --manifest-url-header="" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233817 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233822 2584 flags.go:64] FLAG: --max-open-files="1000000" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233828 2584 flags.go:64] FLAG: --max-pods="110" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233832 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233837 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233843 2584 flags.go:64] FLAG: --memory-manager-policy="None" Sep 10 14:42:22.241321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233847 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233852 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233856 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233861 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233872 2584 flags.go:64] FLAG: --node-status-max-images="50" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233877 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233882 2584 flags.go:64] FLAG: --oom-score-adj="-999" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233886 2584 flags.go:64] FLAG: --pod-cidr="" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233891 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233899 2584 flags.go:64] FLAG: --pod-manifest-path="" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233904 2584 flags.go:64] FLAG: --pod-max-pids="-1" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233909 2584 flags.go:64] FLAG: --pods-per-core="0" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233913 2584 flags.go:64] FLAG: --port="10250" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233918 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233923 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08caa951a9cf88c34" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233928 2584 flags.go:64] FLAG: --qos-reserved="" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233933 2584 flags.go:64] FLAG: --read-only-port="10255" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233939 2584 flags.go:64] FLAG: --register-node="true" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233944 2584 flags.go:64] FLAG: --register-with-taints="" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233949 2584 flags.go:64] FLAG: --registry-burst="10" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233955 2584 flags.go:64] FLAG: --registry-qps="5" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233960 2584 flags.go:64] FLAG: --reserved-cpus="" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233965 2584 flags.go:64] FLAG: --reserved-memory="" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233970 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233975 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 10 14:42:22.242299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233980 2584 flags.go:64] FLAG: --rotate-certificates="false" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233985 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233989 2584 flags.go:64] FLAG: --runonce="false" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233994 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.233999 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234004 2584 flags.go:64] FLAG: --seccomp-default="false" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234009 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234013 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234018 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234023 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234028 2584 flags.go:64] FLAG: --storage-driver-password="root" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234033 2584 flags.go:64] FLAG: --storage-driver-secure="false" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234037 2584 flags.go:64] FLAG: --storage-driver-table="stats" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234042 2584 flags.go:64] FLAG: --storage-driver-user="root" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234046 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234051 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234056 2584 flags.go:64] FLAG: --system-cgroups="" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234061 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234069 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234074 2584 flags.go:64] FLAG: --tls-cert-file="" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234079 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234085 2584 flags.go:64] FLAG: --tls-min-version="" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234089 2584 flags.go:64] FLAG: --tls-private-key-file="" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234094 2584 flags.go:64] FLAG: --topology-manager-policy="none" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234102 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 10 14:42:22.242945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234108 2584 flags.go:64] FLAG: --topology-manager-scope="container" Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234113 2584 flags.go:64] FLAG: --v="2" Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234120 2584 flags.go:64] FLAG: --version="false" Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234126 2584 flags.go:64] FLAG: --vmodule="" Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234132 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.234137 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234312 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234318 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234323 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234327 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234332 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234336 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234340 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234344 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234348 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234351 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234356 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234360 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234364 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234368 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234372 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 14:42:22.243546 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234376 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234381 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234385 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234389 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234396 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234401 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234406 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234411 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234415 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234419 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234424 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234429 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234433 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234439 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234443 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234447 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234451 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234456 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234460 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234464 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 14:42:22.244076 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234468 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234472 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234476 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234480 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234484 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234488 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234492 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234497 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234501 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234505 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234509 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234513 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234517 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234521 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234525 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234529 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234533 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234538 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234542 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234546 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 14:42:22.244558 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234550 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234563 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234570 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234575 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234580 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234587 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234592 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234597 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234602 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234606 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234627 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234632 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234636 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234640 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234644 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234648 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234652 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234657 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234661 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 14:42:22.245096 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234665 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234669 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234674 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234678 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234682 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234685 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234690 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234694 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234698 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234702 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234707 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234711 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234715 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234719 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234723 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234727 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234731 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234740 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234747 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 14:42:22.245547 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234751 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234757 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234763 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234768 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234772 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.234776 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.235255 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.243174 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.243298 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.244132 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.244140 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244196 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244202 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244206 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 14:42:22.246060 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244210 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244213 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244217 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244221 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244224 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244227 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244230 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244234 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244237 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244241 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244244 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244247 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244249 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244252 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244255 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244257 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244260 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244263 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244266 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 14:42:22.246411 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244269 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244272 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244274 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244277 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244280 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244282 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244285 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244289 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244291 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244295 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244298 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244301 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244303 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244306 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244309 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244311 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244314 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244317 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244319 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244322 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 14:42:22.246908 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244324 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244327 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244330 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244332 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244335 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244338 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244340 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244343 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244346 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244348 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244351 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244354 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244356 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244359 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244362 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244365 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244367 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244370 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244373 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244375 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 14:42:22.247390 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244378 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244381 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244383 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244386 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244388 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244391 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244393 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244396 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244399 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244401 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244404 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244406 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244409 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244412 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244414 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244416 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244419 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244422 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244425 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 14:42:22.247881 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244427 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244430 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244432 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244435 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244438 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244441 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244443 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244445 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244448 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244451 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244453 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244456 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244459 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244461 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244464 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244466 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244469 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 14:42:22.248341 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244472 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.244477 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244583 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244588 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244590 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244593 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244595 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244598 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244601 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244603 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244606 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244619 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244622 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244627 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244632 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 10 14:42:22.248769 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244635 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244637 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244640 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244643 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244646 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244648 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244650 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244653 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244656 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244658 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244660 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244663 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244665 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244668 2584 feature_gate.go:328] unrecognized feature gate: Example2 Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244671 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244673 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244676 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244679 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244681 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244684 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 10 14:42:22.249138 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244687 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244689 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244691 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244694 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244696 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244699 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244701 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244704 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244706 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244709 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244711 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244714 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244718 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244721 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244724 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244727 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244729 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244732 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244735 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244738 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 10 14:42:22.249687 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244740 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244743 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244746 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244748 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244751 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244753 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244756 2584 feature_gate.go:328] unrecognized feature gate: Example Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244759 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244761 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244764 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244767 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244771 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244774 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244776 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244779 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244781 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244784 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244786 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244789 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244791 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 10 14:42:22.250168 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244793 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244796 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244799 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244801 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244804 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244807 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244809 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244811 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244814 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244816 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244819 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244821 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244824 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244830 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244832 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244835 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244837 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244840 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244843 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244845 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 10 14:42:22.250679 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244848 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244850 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244853 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244856 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244858 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:22.244861 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.244866 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.244962 2584 server.go:962] "Client rotation is on, will bootstrap in background" Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.247441 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.248247 2584 server.go:1019] "Starting client certificate rotation" Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.248353 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.248923 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 10 14:42:22.251156 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.250713 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 14:42:22.255733 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.255708 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 10 14:42:22.268701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.268681 2584 log.go:25] "Validated CRI v1 runtime API" Sep 10 14:42:22.271634 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.271604 2584 log.go:25] "Validated CRI v1 image API" Sep 10 14:42:22.272536 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.272524 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 10 14:42:22.275993 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.275974 2584 fs.go:135] Filesystem UUIDs: map[3af6bde2-0d31-4c02-9777-cb079cc42b70:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ebad69ab-8092-4729-a110-78a8decd4b0a:/dev/nvme0n1p3] Sep 10 14:42:22.276049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.275993 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 10 14:42:22.279892 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.279872 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 14:42:22.281771 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.281664 2584 manager.go:216] Machine: {Timestamp:2026-09-10 14:42:22.280399178 +0000 UTC m=+0.304013902 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099109 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2db65cea603d85410400ce4cf8d720 SystemUUID:ec2db65c-ea60-3d85-4104-00ce4cf8d720 BootID:1e85be03-73d8-442a-86fc-3a089790e0ce Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:28:08:e5:19:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:28:08:e5:19:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:2a:cc:13:9c:83 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 10 14:42:22.281771 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.281764 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 10 14:42:22.281883 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.281874 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 10 14:42:22.284396 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.284370 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 10 14:42:22.284524 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.284398 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-202.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 10 14:42:22.285459 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.285449 2584 topology_manager.go:138] "Creating topology manager with none policy" Sep 10 14:42:22.285500 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.285462 2584 container_manager_linux.go:309] "Creating device plugin manager" Sep 10 14:42:22.285500 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.285470 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 14:42:22.286230 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.286219 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 10 14:42:22.287290 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.287279 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 10 14:42:22.291331 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291320 2584 state_mem.go:36] "Initialized new in-memory state store" Sep 10 14:42:22.291433 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291424 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 10 14:42:22.291464 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291450 2584 kubelet.go:486] "Attempting to sync node with API server" Sep 10 14:42:22.291464 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291459 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 10 14:42:22.291517 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291470 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 10 14:42:22.291517 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291479 2584 kubelet.go:398] "Adding apiserver pod source" Sep 10 14:42:22.291517 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291488 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 10 14:42:22.291842 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291824 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 14:42:22.291907 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.291846 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 10 14:42:22.293804 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.293783 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 10 14:42:22.295977 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.295963 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 10 14:42:22.296046 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.295993 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 10 14:42:22.296155 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296145 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 10 14:42:22.296193 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296159 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 10 14:42:22.296193 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296165 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 10 14:42:22.296193 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296171 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 10 14:42:22.296193 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296179 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 10 14:42:22.296193 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296185 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 10 14:42:22.296767 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296758 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 10 14:42:22.296767 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296767 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 10 14:42:22.296827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296774 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 10 14:42:22.296827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296779 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 10 14:42:22.296827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296793 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 10 14:42:22.296827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.296801 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 10 14:42:22.297526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.297515 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 10 14:42:22.297526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.297525 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 10 14:42:22.301043 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.301029 2584 server.go:1268] "Started kubelet" Sep 10 14:42:22.301185 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.301130 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 10 14:42:22.301256 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.301212 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 10 14:42:22.301776 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.301739 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 10 14:42:22.301992 ip-10-0-136-202 systemd[1]: Started Kubernetes Kubelet. Sep 10 14:42:22.302196 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.302176 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 10 14:42:22.304935 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.303799 2584 server.go:310] "Adding debug handlers to kubelet server" Sep 10 14:42:22.304935 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.304137 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-202.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 10 14:42:22.304935 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.304762 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 10 14:42:22.306080 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.305345 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 10 14:42:22.306080 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.305415 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2v5dv" Sep 10 14:42:22.307188 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.307172 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 10 14:42:22.307585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.307544 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 10 14:42:22.307719 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.307700 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.307875 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.306950 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-202.ec2.internal.18d3fcb78ec97e4c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-202.ec2.internal,UID:ip-10-0-136-202.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-202.ec2.internal,},FirstTimestamp:2026-09-10 14:42:22.30100334 +0000 UTC m=+0.324618062,LastTimestamp:2026-09-10 14:42:22.30100334 +0000 UTC m=+0.324618062,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-202.ec2.internal,}" Sep 10 14:42:22.308138 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308125 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 10 14:42:22.308315 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308297 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 10 14:42:22.308315 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308315 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 10 14:42:22.308468 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308402 2584 reconstruct.go:97] "Volume reconstruction finished" Sep 10 14:42:22.308468 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308413 2584 reconciler.go:29] "Reconciler: start to sync state" Sep 10 14:42:22.308562 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308494 2584 factory.go:55] Registering systemd factory Sep 10 14:42:22.308562 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308505 2584 factory.go:223] Registration of the systemd container factory successfully Sep 10 14:42:22.308793 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308775 2584 factory.go:153] Registering CRI-O factory Sep 10 14:42:22.308876 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308796 2584 factory.go:223] Registration of the crio container factory successfully Sep 10 14:42:22.308876 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308848 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 10 14:42:22.308876 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308869 2584 factory.go:103] Registering Raw factory Sep 10 14:42:22.309011 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.308882 2584 manager.go:1196] Started watching for new ooms in manager Sep 10 14:42:22.309234 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.309221 2584 manager.go:349] Starting recovery of all containers Sep 10 14:42:22.314069 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.314045 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2v5dv" Sep 10 14:42:22.315020 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.314992 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 10 14:42:22.315103 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.315032 2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 10 14:42:22.315103 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.315038 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-202.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 10 14:42:22.320596 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.320047 2584 manager.go:354] Recovery completed Sep 10 14:42:22.324260 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.324247 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 14:42:22.325835 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.325820 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientMemory" Sep 10 14:42:22.325918 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.325851 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 14:42:22.325918 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.325866 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientPID" Sep 10 14:42:22.326392 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.326375 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 10 14:42:22.326392 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.326389 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 14:42:22.326505 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.326389 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 10 14:42:22.326505 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.326423 2584 state_mem.go:36] "Initialized new in-memory state store" Sep 10 14:42:22.327046 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.327030 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientMemory" Sep 10 14:42:22.327116 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.327056 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 14:42:22.327116 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.327069 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientPID" Sep 10 14:42:22.327289 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.327229 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-202.ec2.internal.18d3fcb7904464a1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-202.ec2.internal,UID:ip-10-0-136-202.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-202.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-202.ec2.internal,},FirstTimestamp:2026-09-10 14:42:22.325834913 +0000 UTC m=+0.349449637,LastTimestamp:2026-09-10 14:42:22.325834913 +0000 UTC m=+0.349449637,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-202.ec2.internal,}" Sep 10 14:42:22.328531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.328515 2584 policy_none.go:49] "None policy: Start" Sep 10 14:42:22.328600 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.328535 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 10 14:42:22.328600 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.328549 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 10 14:42:22.330479 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.330465 2584 policy_none.go:47] "Start" Sep 10 14:42:22.355178 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.355163 2584 manager.go:337] "Starting Device Plugin manager" Sep 10 14:42:22.355317 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.355202 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 10 14:42:22.355317 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.355211 2584 server.go:87] "Starting device plugin registration server" Sep 10 14:42:22.355427 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.355406 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 10 14:42:22.355427 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.355416 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 10 14:42:22.355665 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.355653 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 10 14:42:22.355721 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.355713 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 10 14:42:22.355721 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.355721 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 10 14:42:22.356235 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.356218 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 10 14:42:22.356330 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.356257 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.403544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.403520 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 10 14:42:22.405562 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.404644 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 10 14:42:22.405562 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.404682 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 10 14:42:22.405562 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.404700 2584 kubelet.go:2448] "Starting kubelet main sync loop" Sep 10 14:42:22.405562 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.404735 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 10 14:42:22.406963 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.406948 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 14:42:22.455972 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.455957 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 14:42:22.456773 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.456756 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientMemory" Sep 10 14:42:22.456842 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.456789 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 14:42:22.456842 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.456802 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientPID" Sep 10 14:42:22.456842 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.456828 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.466093 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.466076 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.466138 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.466097 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-202.ec2.internal\": node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.479949 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.479880 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.505216 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.505187 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal"] Sep 10 14:42:22.505287 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.505256 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 14:42:22.506523 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.506510 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientMemory" Sep 10 14:42:22.506629 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.506533 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 14:42:22.506629 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.506544 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientPID" Sep 10 14:42:22.508663 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.508652 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 14:42:22.508778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.508764 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.508820 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.508796 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 14:42:22.508923 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.508908 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7813b39ba37e19e9b5b60931fbe02314-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal\" (UID: \"7813b39ba37e19e9b5b60931fbe02314\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.508964 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.508931 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7813b39ba37e19e9b5b60931fbe02314-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal\" (UID: \"7813b39ba37e19e9b5b60931fbe02314\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.508964 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.508948 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0f8f991da7ffc336926674c917982fec-config\") pod \"kube-apiserver-proxy-ip-10-0-136-202.ec2.internal\" (UID: \"0f8f991da7ffc336926674c917982fec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.509311 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.509284 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientMemory" Sep 10 14:42:22.509311 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.509312 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 14:42:22.509414 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.509336 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientPID" Sep 10 14:42:22.509836 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.509822 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientMemory" Sep 10 14:42:22.509882 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.509846 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 14:42:22.509882 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.509856 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientPID" Sep 10 14:42:22.511438 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.511424 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.511487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.511448 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 10 14:42:22.512090 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.512072 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientMemory" Sep 10 14:42:22.512147 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.512109 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasNoDiskPressure" Sep 10 14:42:22.512147 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.512124 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeHasSufficientPID" Sep 10 14:42:22.537254 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.537234 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-202.ec2.internal\" not found" node="ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.541823 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.541765 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-202.ec2.internal\" not found" node="ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.580266 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.580240 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.609047 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.609024 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0f8f991da7ffc336926674c917982fec-config\") pod \"kube-apiserver-proxy-ip-10-0-136-202.ec2.internal\" (UID: \"0f8f991da7ffc336926674c917982fec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.609117 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.609048 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7813b39ba37e19e9b5b60931fbe02314-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal\" (UID: \"7813b39ba37e19e9b5b60931fbe02314\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.609117 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.609064 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7813b39ba37e19e9b5b60931fbe02314-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal\" (UID: \"7813b39ba37e19e9b5b60931fbe02314\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.609188 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.609129 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7813b39ba37e19e9b5b60931fbe02314-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal\" (UID: \"7813b39ba37e19e9b5b60931fbe02314\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.609188 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.609138 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7813b39ba37e19e9b5b60931fbe02314-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal\" (UID: \"7813b39ba37e19e9b5b60931fbe02314\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.609188 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.609176 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0f8f991da7ffc336926674c917982fec-config\") pod \"kube-apiserver-proxy-ip-10-0-136-202.ec2.internal\" (UID: \"0f8f991da7ffc336926674c917982fec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.681361 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.681336 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.781852 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.781790 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.839386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.839365 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.844886 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:22.844870 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" Sep 10 14:42:22.882406 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.882382 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:22.982824 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:22.982803 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:23.083261 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:23.083216 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-202.ec2.internal\" not found" Sep 10 14:42:23.137552 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.137527 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 14:42:23.208472 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.208448 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" Sep 10 14:42:23.218908 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.218891 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 14:42:23.220668 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.220656 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" Sep 10 14:42:23.226391 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.226379 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 10 14:42:23.248530 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.248504 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 10 14:42:23.248661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.248643 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 14:42:23.248716 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:23.248640 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 10 14:42:24.644213 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.644182 2584 apiserver.go:52] "Watching apiserver" Sep 10 14:42:24.659397 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.659376 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 10 14:42:24.662697 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.662681 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 10 14:42:24.662997 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.662977 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-x27qd","openshift-ovn-kubernetes/ovnkube-node-5z54x","kube-system/global-pull-secret-syncer-zlhxp","openshift-dns/node-resolver-qb7fw","openshift-image-registry/node-ca-6mpq9","openshift-multus/multus-additional-cni-plugins-jfwfh","openshift-network-diagnostics/network-check-target-njbnb","openshift-network-operator/iptables-alerter-k7s8s","kube-system/konnectivity-agent-p2h4g","kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh","openshift-cluster-node-tuning-operator/tuned-99j7k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal","openshift-multus/multus-pc9td"] Sep 10 14:42:24.666320 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.666302 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:24.666395 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.666374 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:24.668321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.668299 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 14:37:23 +0000 UTC" deadline="2028-04-23 20:50:36.812436713 +0000 UTC" Sep 10 14:42:24.668321 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.668320 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14190h8m12.144118921s" Sep 10 14:42:24.668637 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.668626 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.669624 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.669605 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 10 14:42:24.671181 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.671162 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7t6xn\"" Sep 10 14:42:24.671181 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.671174 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 10 14:42:24.671342 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.671168 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 10 14:42:24.671342 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.671253 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 10 14:42:24.671425 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.671372 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.671459 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.671432 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:24.674092 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.674077 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.676469 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.676441 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 10 14:42:24.676526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.676501 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.676572 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.676558 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9p9w6\"" Sep 10 14:42:24.676616 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.676583 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 10 14:42:24.678797 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.678757 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 10 14:42:24.678961 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.678940 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 10 14:42:24.678961 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.678949 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 10 14:42:24.679107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.679024 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qhscm\"" Sep 10 14:42:24.679159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.679138 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.681827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.681798 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q2tx4\"" Sep 10 14:42:24.681952 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.681835 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:24.681952 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.681901 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 10 14:42:24.681952 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.681930 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 10 14:42:24.682535 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.682509 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 10 14:42:24.682535 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.682524 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 10 14:42:24.682672 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.682620 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 10 14:42:24.685622 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.685551 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.685622 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.685577 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hlwsw\"" Sep 10 14:42:24.685777 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.685627 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 10 14:42:24.685777 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.685699 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 10 14:42:24.687169 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.687149 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.687871 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.687856 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 10 14:42:24.688090 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.688078 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wsft2\"" Sep 10 14:42:24.688145 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.688117 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mk7cp" Sep 10 14:42:24.689677 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.689656 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 10 14:42:24.689814 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.689758 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 10 14:42:24.689814 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.689769 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.689814 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.689794 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 10 14:42:24.689814 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.689809 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 10 14:42:24.690036 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.689919 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 10 14:42:24.690099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.690090 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jqkh4\"" Sep 10 14:42:24.691273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.691246 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 10 14:42:24.692501 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.692483 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 10 14:42:24.692501 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.692496 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jxjhx\"" Sep 10 14:42:24.695765 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.692488 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 10 14:42:24.695765 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.692599 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 10 14:42:24.695765 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.693249 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:24.695765 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.693294 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:24.695765 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.695585 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mk7cp" Sep 10 14:42:24.696860 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.696845 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.700483 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.700464 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 10 14:42:24.700483 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.700479 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 10 14:42:24.700632 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.700506 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qvtnm\"" Sep 10 14:42:24.761403 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.761235 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 10 14:42:24.765418 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765398 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65b1d81e-eb65-41f4-b525-4cc3ec113c55-konnectivity-ca\") pod \"konnectivity-agent-p2h4g\" (UID: \"65b1d81e-eb65-41f4-b525-4cc3ec113c55\") " pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:24.765483 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765427 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05fbf59b-3045-4815-aae2-2cae55138ab0-cni-binary-copy\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.765483 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765444 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-kubelet\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.765483 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765461 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01b92fcc-fb48-4ed4-9bf8-86594f042671-kubelet-config\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.765604 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765515 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.765604 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765539 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-slash\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.765604 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765554 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.765732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765596 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-systemd\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.765732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765633 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:24.765732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765661 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-cni-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.765732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765699 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-netns\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765735 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cni-binary-copy\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765756 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrlbk\" (UniqueName: \"kubernetes.io/projected/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-kube-api-access-vrlbk\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765771 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-kubelet\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765786 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-kubernetes\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765807 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-ovn\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765823 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4365ce51-f8bc-4b0f-9864-e9309738013f-ovn-node-metrics-cert\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765853 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-var-lib-kubelet\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.765896 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765882 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765908 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xms7r\" (UniqueName: \"kubernetes.io/projected/3102b3a6-3613-4302-b3ec-6abf6382a595-kube-api-access-xms7r\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765932 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-daemon-config\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765955 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3102b3a6-3613-4302-b3ec-6abf6382a595-host-slash\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.765981 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-run\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766004 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766043 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfcf1f64-077b-471c-8cba-0f0a90180acd-host\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766067 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766099 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysconfig\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766122 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-tuned\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766137 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/416d5edd-4a50-4621-947f-46bffc5cb8c0-tmp-dir\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766158 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-system-cni-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766175 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-hostroot\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766192 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twvns\" (UniqueName: \"kubernetes.io/projected/8a85e3e9-009d-4afd-9b07-235a6fb78af3-kube-api-access-twvns\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:24.766247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766225 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766254 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766271 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-ovnkube-script-lib\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766310 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-multus-certs\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766326 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-system-cni-dir\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766340 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfcf1f64-077b-471c-8cba-0f0a90180acd-serviceca\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766357 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-cni-multus\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766384 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc2wl\" (UniqueName: \"kubernetes.io/projected/05fbf59b-3045-4815-aae2-2cae55138ab0-kube-api-access-zc2wl\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766408 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-systemd-units\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766425 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-etc-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766444 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766457 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-node-log\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766471 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-host\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766489 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-k8s-cni-cncf-io\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766509 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-modprobe-d\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766531 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/416d5edd-4a50-4621-947f-46bffc5cb8c0-hosts-file\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.766680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766547 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3102b3a6-3613-4302-b3ec-6abf6382a595-iptables-alerter-script\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766566 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-var-lib-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766590 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-env-overrides\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766610 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysctl-conf\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766624 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-sys\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766638 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-etc-selinux\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766661 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ch4p\" (UniqueName: \"kubernetes.io/projected/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-kube-api-access-5ch4p\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766678 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpqkf\" (UniqueName: \"kubernetes.io/projected/cfcf1f64-077b-471c-8cba-0f0a90180acd-kube-api-access-hpqkf\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766698 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-systemd\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766718 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-log-socket\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766738 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-device-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766788 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-sys-fs\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766822 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-cni-bin\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766843 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-run-netns\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766868 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-cni-bin\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767104 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766894 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-ovnkube-config\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766912 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1554917c-70ca-4169-80a3-8ebf544e59d5-tmp\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766927 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-cnibin\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-cni-netd\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766963 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysctl-d\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766979 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jjbl\" (UniqueName: \"kubernetes.io/projected/416d5edd-4a50-4621-947f-46bffc5cb8c0-kube-api-access-2jjbl\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.766998 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65b1d81e-eb65-41f4-b525-4cc3ec113c55-agent-certs\") pod \"konnectivity-agent-p2h4g\" (UID: \"65b1d81e-eb65-41f4-b525-4cc3ec113c55\") " pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767014 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-socket-dir-parent\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767031 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-socket-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767069 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-registration-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01b92fcc-fb48-4ed4-9bf8-86594f042671-dbus\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767106 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-conf-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767120 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-etc-kubernetes\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767133 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cnibin\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767148 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2n9c\" (UniqueName: \"kubernetes.io/projected/4365ce51-f8bc-4b0f-9864-e9309738013f-kube-api-access-l2n9c\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767161 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-lib-modules\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.767575 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.767179 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gppq8\" (UniqueName: \"kubernetes.io/projected/1554917c-70ca-4169-80a3-8ebf544e59d5-kube-api-access-gppq8\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.865326 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:24.865289 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8f991da7ffc336926674c917982fec.slice/crio-6951af9d404334919576a8dcb65bf7d86a9032231f2aef539a55b19a02867824 WatchSource:0}: Error finding container 6951af9d404334919576a8dcb65bf7d86a9032231f2aef539a55b19a02867824: Status 404 returned error can't find the container with id 6951af9d404334919576a8dcb65bf7d86a9032231f2aef539a55b19a02867824 Sep 10 14:42:24.865557 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:24.865537 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7813b39ba37e19e9b5b60931fbe02314.slice/crio-e355644d05c05add8e373fa62b090d9b469a5c7f0ad6908c775af7ad17e5b517 WatchSource:0}: Error finding container e355644d05c05add8e373fa62b090d9b469a5c7f0ad6908c775af7ad17e5b517: Status 404 returned error can't find the container with id e355644d05c05add8e373fa62b090d9b469a5c7f0ad6908c775af7ad17e5b517 Sep 10 14:42:24.867506 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867487 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfcf1f64-077b-471c-8cba-0f0a90180acd-host\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.867567 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867515 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.867609 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867565 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysconfig\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.867609 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867588 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-tuned\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.867701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867613 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/416d5edd-4a50-4621-947f-46bffc5cb8c0-tmp-dir\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.867701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867636 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-system-cni-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.867701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867658 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-hostroot\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.867701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867675 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysconfig\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.867701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867681 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvns\" (UniqueName: \"kubernetes.io/projected/8a85e3e9-009d-4afd-9b07-235a6fb78af3-kube-api-access-twvns\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:24.867895 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867704 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.867895 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867720 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cfcf1f64-077b-471c-8cba-0f0a90180acd-host\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.867895 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.867895 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867754 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-ovnkube-script-lib\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.867895 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867780 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-multus-certs\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.867895 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.867781 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-hostroot\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.868128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868087 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-system-cni-dir\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.868128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868120 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfcf1f64-077b-471c-8cba-0f0a90180acd-serviceca\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.868222 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868144 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-cni-multus\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.868222 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868168 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc2wl\" (UniqueName: \"kubernetes.io/projected/05fbf59b-3045-4815-aae2-2cae55138ab0-kube-api-access-zc2wl\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.868222 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868191 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-systemd-units\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868220 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868235 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-etc-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868236 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/416d5edd-4a50-4621-947f-46bffc5cb8c0-tmp-dir\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868258 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868294 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-node-log\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868300 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-multus-certs\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868297 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868327 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-host\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868349 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-k8s-cni-cncf-io\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.868359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868361 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-node-log\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-modprobe-d\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868402 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/416d5edd-4a50-4621-947f-46bffc5cb8c0-hosts-file\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868402 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-cni-multus\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868394 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868428 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3102b3a6-3613-4302-b3ec-6abf6382a595-iptables-alerter-script\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868454 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-var-lib-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868488 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-system-cni-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868493 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-env-overrides\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868515 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysctl-conf\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-sys\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-etc-selinux\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868579 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ch4p\" (UniqueName: \"kubernetes.io/projected/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-kube-api-access-5ch4p\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868601 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpqkf\" (UniqueName: \"kubernetes.io/projected/cfcf1f64-077b-471c-8cba-0f0a90180acd-kube-api-access-hpqkf\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868629 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-systemd\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868653 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-log-socket\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868674 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.868802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868713 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-device-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868737 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-sys-fs\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868782 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-cni-bin\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868822 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-ovnkube-script-lib\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868837 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-run-netns\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868848 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cfcf1f64-077b-471c-8cba-0f0a90180acd-serviceca\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868861 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-cni-bin\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868940 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-system-cni-dir\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.868995 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-etc-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869002 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-systemd-units\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869069 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysctl-conf\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869069 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-log-socket\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869101 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/416d5edd-4a50-4621-947f-46bffc5cb8c0-hosts-file\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869128 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869140 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-etc-selinux\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869144 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-host\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869169 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-device-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869180 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-k8s-cni-cncf-io\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.869498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869233 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-sys-fs\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869265 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-modprobe-d\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869265 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-cni-bin\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869293 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-run-netns\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869305 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-ovnkube-config\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869323 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-env-overrides\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869332 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1554917c-70ca-4169-80a3-8ebf544e59d5-tmp\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869323 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-systemd\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869355 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-cni-bin\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869356 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-cnibin\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869371 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-var-lib-openvswitch\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869416 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-sys\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869420 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-cni-netd\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869459 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-cnibin\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869454 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysctl-d\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869507 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jjbl\" (UniqueName: \"kubernetes.io/projected/416d5edd-4a50-4621-947f-46bffc5cb8c0-kube-api-access-2jjbl\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869578 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-cni-netd\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869606 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65b1d81e-eb65-41f4-b525-4cc3ec113c55-agent-certs\") pod \"konnectivity-agent-p2h4g\" (UID: \"65b1d81e-eb65-41f4-b525-4cc3ec113c55\") " pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:24.870273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-socket-dir-parent\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869637 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-sysctl-d\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869674 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-socket-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869723 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-registration-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869749 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01b92fcc-fb48-4ed4-9bf8-86594f042671-dbus\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869750 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869796 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-conf-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869807 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-registration-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869832 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-etc-kubernetes\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869859 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cnibin\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869858 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3102b3a6-3613-4302-b3ec-6abf6382a595-iptables-alerter-script\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869889 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4365ce51-f8bc-4b0f-9864-e9309738013f-ovnkube-config\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869905 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-etc-kubernetes\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869904 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-socket-dir-parent\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869920 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cnibin\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2n9c\" (UniqueName: \"kubernetes.io/projected/4365ce51-f8bc-4b0f-9864-e9309738013f-kube-api-access-l2n9c\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869972 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-lib-modules\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.871124 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.869997 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01b92fcc-fb48-4ed4-9bf8-86594f042671-dbus\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870008 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-socket-dir\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870043 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-conf-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870080 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gppq8\" (UniqueName: \"kubernetes.io/projected/1554917c-70ca-4169-80a3-8ebf544e59d5-kube-api-access-gppq8\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870130 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-lib-modules\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870183 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65b1d81e-eb65-41f4-b525-4cc3ec113c55-konnectivity-ca\") pod \"konnectivity-agent-p2h4g\" (UID: \"65b1d81e-eb65-41f4-b525-4cc3ec113c55\") " pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870276 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05fbf59b-3045-4815-aae2-2cae55138ab0-cni-binary-copy\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870307 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-kubelet\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870351 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01b92fcc-fb48-4ed4-9bf8-86594f042671-kubelet-config\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870413 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870427 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870440 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-slash\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870464 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870476 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-var-lib-kubelet\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870521 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870563 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01b92fcc-fb48-4ed4-9bf8-86594f042671-kubelet-config\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-systemd\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.872151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870597 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-slash\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870618 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870642 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-systemd\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870645 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-cni-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870668 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-netns\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870691 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cni-binary-copy\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870715 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrlbk\" (UniqueName: \"kubernetes.io/projected/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-kube-api-access-vrlbk\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870738 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-kubelet\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870759 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-kubernetes\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870783 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-ovn\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870805 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4365ce51-f8bc-4b0f-9864-e9309738013f-ovn-node-metrics-cert\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870828 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-var-lib-kubelet\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870851 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870862 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-cni-dir\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870874 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xms7r\" (UniqueName: \"kubernetes.io/projected/3102b3a6-3613-4302-b3ec-6abf6382a595-kube-api-access-xms7r\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870898 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-daemon-config\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870901 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05fbf59b-3045-4815-aae2-2cae55138ab0-host-run-netns\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.872778 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870921 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3102b3a6-3613-4302-b3ec-6abf6382a595-host-slash\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870942 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-run\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870964 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.870977 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05fbf59b-3045-4815-aae2-2cae55138ab0-cni-binary-copy\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.871061 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871109 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.871133 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs podName:8a85e3e9-009d-4afd-9b07-235a6fb78af3 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:25.371108019 +0000 UTC m=+2.043133771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs") pod "network-metrics-daemon-x27qd" (UID: "8a85e3e9-009d-4afd-9b07-235a6fb78af3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871162 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-var-lib-kubelet\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.871190 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.871265 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret podName:01b92fcc-fb48-4ed4-9bf8-86594f042671 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:25.371248647 +0000 UTC m=+2.043274412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret") pod "global-pull-secret-syncer-zlhxp" (UID: "01b92fcc-fb48-4ed4-9bf8-86594f042671") : object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871314 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-host-kubelet\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871341 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-cni-binary-copy\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871352 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/65b1d81e-eb65-41f4-b525-4cc3ec113c55-konnectivity-ca\") pod \"konnectivity-agent-p2h4g\" (UID: \"65b1d81e-eb65-41f4-b525-4cc3ec113c55\") " pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871366 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-kubernetes\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871405 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4365ce51-f8bc-4b0f-9864-e9309738013f-run-ovn\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871444 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3102b3a6-3613-4302-b3ec-6abf6382a595-host-slash\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871448 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1554917c-70ca-4169-80a3-8ebf544e59d5-run\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871770 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/05fbf59b-3045-4815-aae2-2cae55138ab0-multus-daemon-config\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.873531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.871991 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 14:42:24.874098 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.872345 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1554917c-70ca-4169-80a3-8ebf544e59d5-tmp\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.874098 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.872521 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1554917c-70ca-4169-80a3-8ebf544e59d5-etc-tuned\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.874098 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.873413 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/65b1d81e-eb65-41f4-b525-4cc3ec113c55-agent-certs\") pod \"konnectivity-agent-p2h4g\" (UID: \"65b1d81e-eb65-41f4-b525-4cc3ec113c55\") " pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:24.874098 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.873791 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4365ce51-f8bc-4b0f-9864-e9309738013f-ovn-node-metrics-cert\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.876324 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.876259 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twvns\" (UniqueName: \"kubernetes.io/projected/8a85e3e9-009d-4afd-9b07-235a6fb78af3-kube-api-access-twvns\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:24.876851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.876363 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 14:42:24.876851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.876384 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 14:42:24.876851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.876398 2584 projected.go:196] Error preparing data for projected volume kube-api-access-nqgpg for pod openshift-network-diagnostics/network-check-target-njbnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:24.876851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:24.876447 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg podName:9895ec05-d97a-4963-88db-96065abb41f4 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:25.376432358 +0000 UTC m=+2.048458108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nqgpg" (UniqueName: "kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg") pod "network-check-target-njbnb" (UID: "9895ec05-d97a-4963-88db-96065abb41f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:24.877287 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.877265 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc2wl\" (UniqueName: \"kubernetes.io/projected/05fbf59b-3045-4815-aae2-2cae55138ab0-kube-api-access-zc2wl\") pod \"multus-pc9td\" (UID: \"05fbf59b-3045-4815-aae2-2cae55138ab0\") " pod="openshift-multus/multus-pc9td" Sep 10 14:42:24.877579 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.877556 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ch4p\" (UniqueName: \"kubernetes.io/projected/07ccdd2e-1d96-433c-a46d-5d168aca0bd1-kube-api-access-5ch4p\") pod \"aws-ebs-csi-driver-node-f7frh\" (UID: \"07ccdd2e-1d96-433c-a46d-5d168aca0bd1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:24.877894 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.877878 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpqkf\" (UniqueName: \"kubernetes.io/projected/cfcf1f64-077b-471c-8cba-0f0a90180acd-kube-api-access-hpqkf\") pod \"node-ca-6mpq9\" (UID: \"cfcf1f64-077b-471c-8cba-0f0a90180acd\") " pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:24.878106 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.878085 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jjbl\" (UniqueName: \"kubernetes.io/projected/416d5edd-4a50-4621-947f-46bffc5cb8c0-kube-api-access-2jjbl\") pod \"node-resolver-qb7fw\" (UID: \"416d5edd-4a50-4621-947f-46bffc5cb8c0\") " pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:24.878744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.878726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gppq8\" (UniqueName: \"kubernetes.io/projected/1554917c-70ca-4169-80a3-8ebf544e59d5-kube-api-access-gppq8\") pod \"tuned-99j7k\" (UID: \"1554917c-70ca-4169-80a3-8ebf544e59d5\") " pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:24.878931 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.878916 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2n9c\" (UniqueName: \"kubernetes.io/projected/4365ce51-f8bc-4b0f-9864-e9309738013f-kube-api-access-l2n9c\") pod \"ovnkube-node-5z54x\" (UID: \"4365ce51-f8bc-4b0f-9864-e9309738013f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:24.880235 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.880218 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xms7r\" (UniqueName: \"kubernetes.io/projected/3102b3a6-3613-4302-b3ec-6abf6382a595-kube-api-access-xms7r\") pod \"iptables-alerter-k7s8s\" (UID: \"3102b3a6-3613-4302-b3ec-6abf6382a595\") " pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:24.880318 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:24.880301 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrlbk\" (UniqueName: \"kubernetes.io/projected/11e4848c-8666-4f5a-bc95-d0e5c8cc5154-kube-api-access-vrlbk\") pod \"multus-additional-cni-plugins-jfwfh\" (UID: \"11e4848c-8666-4f5a-bc95-d0e5c8cc5154\") " pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:25.002538 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.002472 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" Sep 10 14:42:25.012141 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.012118 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ccdd2e_1d96_433c_a46d_5d168aca0bd1.slice/crio-2ab0fac734a6cdf4a7ffb6d5266100f7adbaf5e93495967700b8a7d3891f9865 WatchSource:0}: Error finding container 2ab0fac734a6cdf4a7ffb6d5266100f7adbaf5e93495967700b8a7d3891f9865: Status 404 returned error can't find the container with id 2ab0fac734a6cdf4a7ffb6d5266100f7adbaf5e93495967700b8a7d3891f9865 Sep 10 14:42:25.015824 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.015803 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qb7fw" Sep 10 14:42:25.022258 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.022241 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6mpq9" Sep 10 14:42:25.023289 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.023270 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416d5edd_4a50_4621_947f_46bffc5cb8c0.slice/crio-56ae298042d5c04e127633a453c84556fe2ba35a2d010b6d3c4b05060f391568 WatchSource:0}: Error finding container 56ae298042d5c04e127633a453c84556fe2ba35a2d010b6d3c4b05060f391568: Status 404 returned error can't find the container with id 56ae298042d5c04e127633a453c84556fe2ba35a2d010b6d3c4b05060f391568 Sep 10 14:42:25.031394 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.031373 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfcf1f64_077b_471c_8cba_0f0a90180acd.slice/crio-91abeee0462fc71a9f255d88029eb0aa90ebcd527bd6fd6118d06726d2a9dae8 WatchSource:0}: Error finding container 91abeee0462fc71a9f255d88029eb0aa90ebcd527bd6fd6118d06726d2a9dae8: Status 404 returned error can't find the container with id 91abeee0462fc71a9f255d88029eb0aa90ebcd527bd6fd6118d06726d2a9dae8 Sep 10 14:42:25.034107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.034092 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" Sep 10 14:42:25.041374 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.041354 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11e4848c_8666_4f5a_bc95_d0e5c8cc5154.slice/crio-2c5cca5ad213564bd5face50249d88b8d977e7ca8c50df4c148820f2bf64c41f WatchSource:0}: Error finding container 2c5cca5ad213564bd5face50249d88b8d977e7ca8c50df4c148820f2bf64c41f: Status 404 returned error can't find the container with id 2c5cca5ad213564bd5face50249d88b8d977e7ca8c50df4c148820f2bf64c41f Sep 10 14:42:25.049283 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.049265 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:25.056565 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.056546 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b1d81e_eb65_41f4_b525_4cc3ec113c55.slice/crio-681b2b1b690e7f01c0c9a8e7850bc2f0d0a90e6d833927c8a97286d623968ae7 WatchSource:0}: Error finding container 681b2b1b690e7f01c0c9a8e7850bc2f0d0a90e6d833927c8a97286d623968ae7: Status 404 returned error can't find the container with id 681b2b1b690e7f01c0c9a8e7850bc2f0d0a90e6d833927c8a97286d623968ae7 Sep 10 14:42:25.061405 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.061390 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pc9td" Sep 10 14:42:25.068665 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.068647 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05fbf59b_3045_4815_aae2_2cae55138ab0.slice/crio-b35c519ed1073abd2049dfbb2558fb6082d0cc81d28efe5e6d718b9858712939 WatchSource:0}: Error finding container b35c519ed1073abd2049dfbb2558fb6082d0cc81d28efe5e6d718b9858712939: Status 404 returned error can't find the container with id b35c519ed1073abd2049dfbb2558fb6082d0cc81d28efe5e6d718b9858712939 Sep 10 14:42:25.087619 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.087594 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:25.087994 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.087973 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 14:42:25.094784 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.094765 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4365ce51_f8bc_4b0f_9864_e9309738013f.slice/crio-7e80a2921e839ef43d934827f8b65f4c63c3951000b67fec69cf01e80985651c WatchSource:0}: Error finding container 7e80a2921e839ef43d934827f8b65f4c63c3951000b67fec69cf01e80985651c: Status 404 returned error can't find the container with id 7e80a2921e839ef43d934827f8b65f4c63c3951000b67fec69cf01e80985651c Sep 10 14:42:25.100583 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.100566 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k7s8s" Sep 10 14:42:25.104963 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.104947 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-99j7k" Sep 10 14:42:25.108447 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.108427 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3102b3a6_3613_4302_b3ec_6abf6382a595.slice/crio-b00d6313d05ce4cd0d691f3f3de3abded48bff9bdb9f15504a558a237ac7e3ce WatchSource:0}: Error finding container b00d6313d05ce4cd0d691f3f3de3abded48bff9bdb9f15504a558a237ac7e3ce: Status 404 returned error can't find the container with id b00d6313d05ce4cd0d691f3f3de3abded48bff9bdb9f15504a558a237ac7e3ce Sep 10 14:42:25.113123 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:25.113107 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1554917c_70ca_4169_80a3_8ebf544e59d5.slice/crio-1ffed071fc1e7ab29d61b41758a22ae126b51d04ec748fba2858d222f389ceb1 WatchSource:0}: Error finding container 1ffed071fc1e7ab29d61b41758a22ae126b51d04ec748fba2858d222f389ceb1: Status 404 returned error can't find the container with id 1ffed071fc1e7ab29d61b41758a22ae126b51d04ec748fba2858d222f389ceb1 Sep 10 14:42:25.207801 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.207775 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 14:42:25.374511 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.373758 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:25.374511 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.373805 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:25.374511 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.373937 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:25.374511 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.373995 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs podName:8a85e3e9-009d-4afd-9b07-235a6fb78af3 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:26.37397757 +0000 UTC m=+3.046003321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs") pod "network-metrics-daemon-x27qd" (UID: "8a85e3e9-009d-4afd-9b07-235a6fb78af3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:25.374511 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.374388 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:25.374511 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.374435 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret podName:01b92fcc-fb48-4ed4-9bf8-86594f042671 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:26.374420611 +0000 UTC m=+3.046446364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret") pod "global-pull-secret-syncer-zlhxp" (UID: "01b92fcc-fb48-4ed4-9bf8-86594f042671") : object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:25.474837 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.474797 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:25.475005 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.474934 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 14:42:25.475005 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.474949 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 14:42:25.475005 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.474957 2584 projected.go:196] Error preparing data for projected volume kube-api-access-nqgpg for pod openshift-network-diagnostics/network-check-target-njbnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:25.475005 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.474998 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg podName:9895ec05-d97a-4963-88db-96065abb41f4 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:26.474985447 +0000 UTC m=+3.147011210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqgpg" (UniqueName: "kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg") pod "network-check-target-njbnb" (UID: "9895ec05-d97a-4963-88db-96065abb41f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:25.699751 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.699498 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:37:24 +0000 UTC" deadline="2028-03-08 13:36:37.84323434 +0000 UTC" Sep 10 14:42:25.699751 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.699565 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13078h54m12.143673896s" Sep 10 14:42:25.760026 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.759733 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:25.760026 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:25.759863 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:25.792627 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.792531 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p2h4g" event={"ID":"65b1d81e-eb65-41f4-b525-4cc3ec113c55","Type":"ContainerStarted","Data":"681b2b1b690e7f01c0c9a8e7850bc2f0d0a90e6d833927c8a97286d623968ae7"} Sep 10 14:42:25.811701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.811627 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6mpq9" event={"ID":"cfcf1f64-077b-471c-8cba-0f0a90180acd","Type":"ContainerStarted","Data":"91abeee0462fc71a9f255d88029eb0aa90ebcd527bd6fd6118d06726d2a9dae8"} Sep 10 14:42:25.814230 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.814019 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 14:42:25.833114 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.832902 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qb7fw" event={"ID":"416d5edd-4a50-4621-947f-46bffc5cb8c0","Type":"ContainerStarted","Data":"56ae298042d5c04e127633a453c84556fe2ba35a2d010b6d3c4b05060f391568"} Sep 10 14:42:25.862671 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.860360 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" event={"ID":"07ccdd2e-1d96-433c-a46d-5d168aca0bd1","Type":"ContainerStarted","Data":"2ab0fac734a6cdf4a7ffb6d5266100f7adbaf5e93495967700b8a7d3891f9865"} Sep 10 14:42:25.879161 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.879130 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" event={"ID":"7813b39ba37e19e9b5b60931fbe02314","Type":"ContainerStarted","Data":"e355644d05c05add8e373fa62b090d9b469a5c7f0ad6908c775af7ad17e5b517"} Sep 10 14:42:25.889944 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.889896 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" event={"ID":"0f8f991da7ffc336926674c917982fec","Type":"ContainerStarted","Data":"6951af9d404334919576a8dcb65bf7d86a9032231f2aef539a55b19a02867824"} Sep 10 14:42:25.916013 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.915981 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-99j7k" event={"ID":"1554917c-70ca-4169-80a3-8ebf544e59d5","Type":"ContainerStarted","Data":"1ffed071fc1e7ab29d61b41758a22ae126b51d04ec748fba2858d222f389ceb1"} Sep 10 14:42:25.929905 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.929875 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k7s8s" event={"ID":"3102b3a6-3613-4302-b3ec-6abf6382a595","Type":"ContainerStarted","Data":"b00d6313d05ce4cd0d691f3f3de3abded48bff9bdb9f15504a558a237ac7e3ce"} Sep 10 14:42:25.959479 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.959325 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pc9td" event={"ID":"05fbf59b-3045-4815-aae2-2cae55138ab0","Type":"ContainerStarted","Data":"b35c519ed1073abd2049dfbb2558fb6082d0cc81d28efe5e6d718b9858712939"} Sep 10 14:42:25.973741 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.973710 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerStarted","Data":"2c5cca5ad213564bd5face50249d88b8d977e7ca8c50df4c148820f2bf64c41f"} Sep 10 14:42:25.991165 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:25.991140 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"7e80a2921e839ef43d934827f8b65f4c63c3951000b67fec69cf01e80985651c"} Sep 10 14:42:26.382889 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:26.382780 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:26.382889 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:26.382828 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:26.383110 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.382974 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:26.383110 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.383075 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs podName:8a85e3e9-009d-4afd-9b07-235a6fb78af3 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:28.383017226 +0000 UTC m=+5.055042981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs") pod "network-metrics-daemon-x27qd" (UID: "8a85e3e9-009d-4afd-9b07-235a6fb78af3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:26.383608 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.383586 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:26.383711 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.383642 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret podName:01b92fcc-fb48-4ed4-9bf8-86594f042671 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:28.383627871 +0000 UTC m=+5.055653627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret") pod "global-pull-secret-syncer-zlhxp" (UID: "01b92fcc-fb48-4ed4-9bf8-86594f042671") : object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:26.483913 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:26.483876 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:26.484103 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.484087 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 14:42:26.484211 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.484114 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 14:42:26.484211 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.484127 2584 projected.go:196] Error preparing data for projected volume kube-api-access-nqgpg for pod openshift-network-diagnostics/network-check-target-njbnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:26.484211 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.484181 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg podName:9895ec05-d97a-4963-88db-96065abb41f4 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:28.484164861 +0000 UTC m=+5.156190626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqgpg" (UniqueName: "kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg") pod "network-check-target-njbnb" (UID: "9895ec05-d97a-4963-88db-96065abb41f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:26.700775 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:26.700660 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:37:24 +0000 UTC" deadline="2028-03-01 19:45:59.825316609 +0000 UTC" Sep 10 14:42:26.700775 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:26.700700 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12917h3m33.124621466s" Sep 10 14:42:26.757762 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:26.756918 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:26.757762 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.757043 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:26.757762 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:26.757420 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:26.757762 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:26.757523 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:27.659409 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:27.659378 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 10 14:42:27.757317 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:27.757289 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:27.757744 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:27.757421 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:28.400468 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:28.399719 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:28.400468 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:28.399769 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:28.400468 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.399897 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:28.400468 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.399958 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs podName:8a85e3e9-009d-4afd-9b07-235a6fb78af3 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:32.399940131 +0000 UTC m=+9.071965904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs") pod "network-metrics-daemon-x27qd" (UID: "8a85e3e9-009d-4afd-9b07-235a6fb78af3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:28.400468 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.400027 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:28.400468 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.400059 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret podName:01b92fcc-fb48-4ed4-9bf8-86594f042671 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:32.400049172 +0000 UTC m=+9.072074928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret") pod "global-pull-secret-syncer-zlhxp" (UID: "01b92fcc-fb48-4ed4-9bf8-86594f042671") : object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:28.500314 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:28.500279 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:28.500489 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.500467 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 14:42:28.500550 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.500498 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 14:42:28.500550 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.500512 2584 projected.go:196] Error preparing data for projected volume kube-api-access-nqgpg for pod openshift-network-diagnostics/network-check-target-njbnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:28.500658 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.500578 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg podName:9895ec05-d97a-4963-88db-96065abb41f4 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:32.500557943 +0000 UTC m=+9.172583716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqgpg" (UniqueName: "kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg") pod "network-check-target-njbnb" (UID: "9895ec05-d97a-4963-88db-96065abb41f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:28.757334 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:28.757256 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:28.757772 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.757402 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:28.757772 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:28.757761 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:28.757879 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:28.757852 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:29.760034 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:29.759522 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:29.760034 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:29.759655 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:30.757573 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:30.757538 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:30.757725 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:30.757542 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:30.757725 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:30.757684 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:30.757789 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:30.757745 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:31.756913 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:31.756875 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:31.757364 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:31.756999 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:32.436590 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:32.436551 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:32.436765 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:32.436607 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:32.436814 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.436764 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:32.436848 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.436827 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs podName:8a85e3e9-009d-4afd-9b07-235a6fb78af3 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:40.436807557 +0000 UTC m=+17.108833313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs") pod "network-metrics-daemon-x27qd" (UID: "8a85e3e9-009d-4afd-9b07-235a6fb78af3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:32.437301 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.437266 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:32.437396 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.437328 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret podName:01b92fcc-fb48-4ed4-9bf8-86594f042671 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:40.437312445 +0000 UTC m=+17.109338214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret") pod "global-pull-secret-syncer-zlhxp" (UID: "01b92fcc-fb48-4ed4-9bf8-86594f042671") : object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:32.537699 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:32.537577 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:32.537873 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.537738 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 14:42:32.537873 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.537759 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 14:42:32.537873 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.537769 2584 projected.go:196] Error preparing data for projected volume kube-api-access-nqgpg for pod openshift-network-diagnostics/network-check-target-njbnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:32.537873 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.537816 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg podName:9895ec05-d97a-4963-88db-96065abb41f4 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:40.537804033 +0000 UTC m=+17.209829805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqgpg" (UniqueName: "kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg") pod "network-check-target-njbnb" (UID: "9895ec05-d97a-4963-88db-96065abb41f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:32.757017 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:32.756936 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:32.757017 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:32.756947 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:32.757548 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.757071 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:32.757548 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:32.757311 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:33.759117 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:33.758879 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:33.759571 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:33.759072 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:34.757555 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:34.757466 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:34.757723 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:34.757610 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:34.757723 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:34.757667 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:34.757840 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:34.757795 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:35.757428 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:35.757398 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:35.757882 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:35.757523 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:36.757427 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:36.757394 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:36.757619 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:36.757516 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:36.757619 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:36.757573 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:36.757979 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:36.757687 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:37.756661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:37.756629 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:37.756816 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:37.756757 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:38.756646 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:38.756611 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:38.757052 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:38.756626 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:38.757052 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:38.756746 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:38.757052 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:38.756790 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:39.757190 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:39.757124 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:39.757602 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:39.757235 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:40.497555 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:40.497515 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:40.497727 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:40.497615 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:40.497727 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.497667 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:40.497727 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.497715 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:40.497872 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.497738 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs podName:8a85e3e9-009d-4afd-9b07-235a6fb78af3 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:56.497722299 +0000 UTC m=+33.169748052 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs") pod "network-metrics-daemon-x27qd" (UID: "8a85e3e9-009d-4afd-9b07-235a6fb78af3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:40.497872 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.497776 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret podName:01b92fcc-fb48-4ed4-9bf8-86594f042671 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:56.497748474 +0000 UTC m=+33.169774228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret") pod "global-pull-secret-syncer-zlhxp" (UID: "01b92fcc-fb48-4ed4-9bf8-86594f042671") : object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:40.598761 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:40.598732 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:40.598949 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.598930 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 14:42:40.599005 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.598959 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 14:42:40.599005 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.598973 2584 projected.go:196] Error preparing data for projected volume kube-api-access-nqgpg for pod openshift-network-diagnostics/network-check-target-njbnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:40.599077 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.599031 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg podName:9895ec05-d97a-4963-88db-96065abb41f4 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:56.599013348 +0000 UTC m=+33.271039102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqgpg" (UniqueName: "kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg") pod "network-check-target-njbnb" (UID: "9895ec05-d97a-4963-88db-96065abb41f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:40.757409 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:40.757333 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:40.757851 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:40.757339 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:40.757851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.757529 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:40.757851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:40.757608 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:41.756982 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:41.756948 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:41.757147 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:41.757060 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:42.757114 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:42.757083 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:42.757567 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:42.757116 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:42.757567 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:42.757216 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:42.757567 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:42.757315 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:43.757696 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:43.757510 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:43.757970 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:43.757780 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:44.048246 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:44.048191 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" event={"ID":"0f8f991da7ffc336926674c917982fec","Type":"ContainerStarted","Data":"a86cb4e7933849a9d7215dadc388259d78c125ff717896924feab45d5bf5e94a"} Sep 10 14:42:44.049962 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:44.049783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-99j7k" event={"ID":"1554917c-70ca-4169-80a3-8ebf544e59d5","Type":"ContainerStarted","Data":"77ef38d1e57923f246f64fc390b1d988fd34aa7aafdcd5312f75b5fc6076680a"} Sep 10 14:42:44.056920 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:44.056898 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"58859ab33f7d28ca2a25f1d50d470ea9946a04ef3cb49778875e6dc5ceb4b11f"} Sep 10 14:42:44.057010 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:44.056999 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"372b55be2344e607139c0f9c43a88b959cdbb4c748b8ed85c0d87bc5da05de2c"} Sep 10 14:42:44.062272 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:44.062225 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-202.ec2.internal" podStartSLOduration=20.062212277 podStartE2EDuration="20.062212277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:44.061802304 +0000 UTC m=+20.733828078" watchObservedRunningTime="2026-09-10 14:42:44.062212277 +0000 UTC m=+20.734238046" Sep 10 14:42:44.757316 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:44.757112 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:44.757428 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:44.757112 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:44.757428 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:44.757358 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:44.757491 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:44.757423 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:45.062678 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.062654 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:42:45.063368 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.063061 2584 generic.go:356] "Generic (PLEG): container finished" podID="4365ce51-f8bc-4b0f-9864-e9309738013f" containerID="58859ab33f7d28ca2a25f1d50d470ea9946a04ef3cb49778875e6dc5ceb4b11f" exitCode=1 Sep 10 14:42:45.063368 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.063129 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"dae05884c5cbda4b90036b5eb6214d1042d0310c6fc48fb94b16d9b521603607"} Sep 10 14:42:45.063368 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.063310 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerDied","Data":"58859ab33f7d28ca2a25f1d50d470ea9946a04ef3cb49778875e6dc5ceb4b11f"} Sep 10 14:42:45.063519 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.063397 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"c0b4aae321470f8d1c9c53f38cdf65cc417c5e30cbaad110f18ad92eb08cbd03"} Sep 10 14:42:45.063519 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.063446 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"bf1790ed88b931242ee421a1a925a9996f834385833d6a771a4b0fa7fda9f42e"} Sep 10 14:42:45.063519 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.063496 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"1c3c17ee24709019a1aa75ad8f65dee17c357217077587b3828a87ec352a9aa2"} Sep 10 14:42:45.064670 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.064647 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p2h4g" event={"ID":"65b1d81e-eb65-41f4-b525-4cc3ec113c55","Type":"ContainerStarted","Data":"8ce5c60f8cc7b79af27e6be7e4d1a7401a830f2eedec393f6276efa02341b484"} Sep 10 14:42:45.066254 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.066231 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6mpq9" event={"ID":"cfcf1f64-077b-471c-8cba-0f0a90180acd","Type":"ContainerStarted","Data":"71ac9c41ed298ba0bfee1c9d24750ec0582ad569e3f164a6724ec51d4c977cc1"} Sep 10 14:42:45.067723 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.067695 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qb7fw" event={"ID":"416d5edd-4a50-4621-947f-46bffc5cb8c0","Type":"ContainerStarted","Data":"1acff721d1e8eefaf1352dd92d86ab8147d8539703df8fb70d77483326cdd7aa"} Sep 10 14:42:45.069307 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.069283 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" event={"ID":"07ccdd2e-1d96-433c-a46d-5d168aca0bd1","Type":"ContainerStarted","Data":"b142181b9dfc494b8b356f10e3adc0760cd148eb4769d856defeb20fb1213938"} Sep 10 14:42:45.070960 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.070930 2584 generic.go:356] "Generic (PLEG): container finished" podID="7813b39ba37e19e9b5b60931fbe02314" containerID="c9bda8b6c66417869618251972edcefba5671ca179fbd70f2508f56beb3afca4" exitCode=0 Sep 10 14:42:45.071053 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.071009 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" event={"ID":"7813b39ba37e19e9b5b60931fbe02314","Type":"ContainerDied","Data":"c9bda8b6c66417869618251972edcefba5671ca179fbd70f2508f56beb3afca4"} Sep 10 14:42:45.074805 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.074783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pc9td" event={"ID":"05fbf59b-3045-4815-aae2-2cae55138ab0","Type":"ContainerStarted","Data":"6c8e03712ee2f3988149c0d63c8d1af5ba3972fa746c658d0ec01a20f1cd66e5"} Sep 10 14:42:45.076508 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.076486 2584 generic.go:356] "Generic (PLEG): container finished" podID="11e4848c-8666-4f5a-bc95-d0e5c8cc5154" containerID="ea609ef63c91ef7956a7a3fa7c5ec8842a260ff47a2ba66896de3e181239ecd7" exitCode=0 Sep 10 14:42:45.076591 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.076523 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerDied","Data":"ea609ef63c91ef7956a7a3fa7c5ec8842a260ff47a2ba66896de3e181239ecd7"} Sep 10 14:42:45.082297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.081982 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p2h4g" podStartSLOduration=3.799965288 podStartE2EDuration="22.081965682s" totalImagesPullingTime="18.282000394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:45.081288486 +0000 UTC m=+21.753314260" watchObservedRunningTime="2026-09-10 14:42:45.081965682 +0000 UTC m=+21.753991455" Sep 10 14:42:45.082297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.082181 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-99j7k" podStartSLOduration=3.77684857 podStartE2EDuration="22.082174219s" totalImagesPullingTime="18.305325649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:44.077782502 +0000 UTC m=+20.749808273" watchObservedRunningTime="2026-09-10 14:42:45.082174219 +0000 UTC m=+21.754199991" Sep 10 14:42:45.116805 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.116616 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pc9td" podStartSLOduration=2.950533133 podStartE2EDuration="22.116600881s" totalImagesPullingTime="19.166067748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:45.115997527 +0000 UTC m=+21.788023300" watchObservedRunningTime="2026-09-10 14:42:45.116600881 +0000 UTC m=+21.788626654" Sep 10 14:42:45.130458 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.130416 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k7s8s" podStartSLOduration=3.820861265 podStartE2EDuration="22.130406399s" totalImagesPullingTime="18.309545134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:45.130318889 +0000 UTC m=+21.802344662" watchObservedRunningTime="2026-09-10 14:42:45.130406399 +0000 UTC m=+21.802432170" Sep 10 14:42:45.143754 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.143700 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qb7fw" podStartSLOduration=3.828542306 podStartE2EDuration="22.143687298s" totalImagesPullingTime="18.315144992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:45.143633977 +0000 UTC m=+21.815659750" watchObservedRunningTime="2026-09-10 14:42:45.143687298 +0000 UTC m=+21.815713070" Sep 10 14:42:45.174795 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.174754 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6mpq9" podStartSLOduration=3.7903689910000002 podStartE2EDuration="22.1747354s" totalImagesPullingTime="18.384366409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:45.159690465 +0000 UTC m=+21.831716238" watchObservedRunningTime="2026-09-10 14:42:45.1747354 +0000 UTC m=+21.846761176" Sep 10 14:42:45.557320 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.557295 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 10 14:42:45.720427 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.720253 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T14:42:45.557315601Z","UUID":"a354620f-23a0-464d-8597-669d7ca3ae13","Handler":null,"Name":"","Endpoint":""} Sep 10 14:42:45.722169 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.722137 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 10 14:42:45.722325 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.722184 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 10 14:42:45.756552 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:45.756526 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:45.756717 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:45.756635 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:46.081413 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:46.081325 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" event={"ID":"07ccdd2e-1d96-433c-a46d-5d168aca0bd1","Type":"ContainerStarted","Data":"7a00f77310d559232af6a8ffc51ba76c3cbfcaea12059c3758e0d63f86833960"} Sep 10 14:42:46.083588 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:46.083563 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" event={"ID":"7813b39ba37e19e9b5b60931fbe02314","Type":"ContainerStarted","Data":"ca98dc0f2fcd1914944d9680762643b8a27d4e45d1d83fd9d405d168d9568db4"} Sep 10 14:42:46.085745 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:46.085667 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k7s8s" event={"ID":"3102b3a6-3613-4302-b3ec-6abf6382a595","Type":"ContainerStarted","Data":"1e371eaf54299d3065a5ba792b405b6498e25c102e7b0794da29a073bf9dff50"} Sep 10 14:42:46.098723 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:46.098679 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-202.ec2.internal" podStartSLOduration=22.098662776 podStartE2EDuration="22.098662776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:46.098639435 +0000 UTC m=+22.770665207" watchObservedRunningTime="2026-09-10 14:42:46.098662776 +0000 UTC m=+22.770688549" Sep 10 14:42:46.756969 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:46.756937 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:46.757148 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:46.756938 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:46.757148 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:46.757044 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:46.757148 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:46.757118 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:47.091361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:47.091296 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:42:47.091921 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:47.091707 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"54267a658b47ea8a56333818b9f02d94f1e1f8beca82b898bfbc98e5d043efc3"} Sep 10 14:42:47.093926 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:47.093903 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" event={"ID":"07ccdd2e-1d96-433c-a46d-5d168aca0bd1","Type":"ContainerStarted","Data":"ad5c9c51bba8d22f9e466efefd3badb1522b4ba0cf8ce77a77c6d042b7fb2ebf"} Sep 10 14:42:47.114473 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:47.114431 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-f7frh" podStartSLOduration=4.103538686 podStartE2EDuration="24.11441963s" totalImagesPullingTime="20.010880944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:47.11425563 +0000 UTC m=+23.786281402" watchObservedRunningTime="2026-09-10 14:42:47.11441963 +0000 UTC m=+23.786445402" Sep 10 14:42:47.757310 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:47.757274 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:47.757569 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:47.757376 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:47.834848 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:47.834808 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:47.835416 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:47.835397 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:48.096239 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:48.096141 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:48.096635 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:48.096529 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p2h4g" Sep 10 14:42:48.757452 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:48.757373 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:48.757452 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:48.757425 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:48.757643 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:48.757519 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:48.757702 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:48.757656 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:49.756922 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:49.756895 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:49.757399 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:49.757022 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:50.104072 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:50.103912 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:42:50.104504 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:50.104477 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"f72240118d7bbee10d7f9137cdb6b4c65a08a621e34d0912f5739f3b497cf694"} Sep 10 14:42:50.104836 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:50.104820 2584 scope.go:117] "RemoveContainer" containerID="58859ab33f7d28ca2a25f1d50d470ea9946a04ef3cb49778875e6dc5ceb4b11f" Sep 10 14:42:50.756443 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:50.756417 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:50.756594 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:50.756417 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:50.756594 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:50.756513 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:50.756665 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:50.756592 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:51.108616 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.108540 2584 generic.go:356] "Generic (PLEG): container finished" podID="11e4848c-8666-4f5a-bc95-d0e5c8cc5154" containerID="6cec0ff186d57a6849e4a3be3927b90607ea001c700c5e4ee7708d41cf4014b2" exitCode=0 Sep 10 14:42:51.109439 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.108629 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerDied","Data":"6cec0ff186d57a6849e4a3be3927b90607ea001c700c5e4ee7708d41cf4014b2"} Sep 10 14:42:51.112444 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.112426 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:42:51.112788 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.112771 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" event={"ID":"4365ce51-f8bc-4b0f-9864-e9309738013f","Type":"ContainerStarted","Data":"ed20f354d67bda54640434890035af4117836be997ad56385e0c0b30b90e3069"} Sep 10 14:42:51.112978 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.112962 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:51.113045 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.113032 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:51.113099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.113070 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:51.129544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.129522 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:51.129655 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.129646 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:42:51.163123 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.163086 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" podStartSLOduration=9.803887087 podStartE2EDuration="28.163071478s" totalImagesPullingTime="18.359184391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:42:51.162939978 +0000 UTC m=+27.834965750" watchObservedRunningTime="2026-09-10 14:42:51.163071478 +0000 UTC m=+27.835097249" Sep 10 14:42:51.757097 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.756933 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:51.757288 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:51.757178 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:51.933040 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.933015 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-njbnb"] Sep 10 14:42:51.933739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.933717 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zlhxp"] Sep 10 14:42:51.933844 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.933830 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:51.933956 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:51.933937 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:51.935436 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.935414 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x27qd"] Sep 10 14:42:51.935537 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:51.935523 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:51.935645 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:51.935626 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:52.117684 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:52.117611 2584 generic.go:356] "Generic (PLEG): container finished" podID="11e4848c-8666-4f5a-bc95-d0e5c8cc5154" containerID="03bddb67d1b62f9fa9118700b5446e71541fdfcb6970cc3b433fec4acae3ef7a" exitCode=0 Sep 10 14:42:52.117684 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:52.117675 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:52.118108 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:52.117702 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerDied","Data":"03bddb67d1b62f9fa9118700b5446e71541fdfcb6970cc3b433fec4acae3ef7a"} Sep 10 14:42:52.118108 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:52.117980 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:53.121658 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:53.121631 2584 generic.go:356] "Generic (PLEG): container finished" podID="11e4848c-8666-4f5a-bc95-d0e5c8cc5154" containerID="f63c2ae2766ae8709339f273725c3e2539b53554ffd220f84792dc08edffe043" exitCode=0 Sep 10 14:42:53.121990 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:53.121714 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerDied","Data":"f63c2ae2766ae8709339f273725c3e2539b53554ffd220f84792dc08edffe043"} Sep 10 14:42:53.757623 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:53.757592 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:53.757804 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:53.757699 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:53.757804 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:53.757735 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:53.757804 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:53.757730 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:53.757804 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:53.757787 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:53.757987 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:53.757844 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:55.757300 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:55.757123 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:55.757665 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:55.757123 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:55.757665 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:55.757376 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-njbnb" podUID="9895ec05-d97a-4963-88db-96065abb41f4" Sep 10 14:42:55.757665 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:55.757475 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zlhxp" podUID="01b92fcc-fb48-4ed4-9bf8-86594f042671" Sep 10 14:42:55.757665 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:55.757140 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:55.757665 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:55.757587 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x27qd" podUID="8a85e3e9-009d-4afd-9b07-235a6fb78af3" Sep 10 14:42:56.191117 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.191089 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-202.ec2.internal" event="NodeReady" Sep 10 14:42:56.191300 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.191215 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 10 14:42:56.224838 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.224811 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85488b788b-pv49c"] Sep 10 14:42:56.312558 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.312528 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m5fkb"] Sep 10 14:42:56.312692 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.312673 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.315990 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.315784 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 10 14:42:56.315990 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.315931 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 10 14:42:56.317591 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.317566 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 10 14:42:56.317764 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.317719 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gb7kw\"" Sep 10 14:42:56.322643 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.322620 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 10 14:42:56.336171 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.336150 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6jvfq"] Sep 10 14:42:56.336310 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.336290 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.339096 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.339077 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 10 14:42:56.339215 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.339078 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mkqg5\"" Sep 10 14:42:56.339215 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.339110 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 10 14:42:56.355052 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.355035 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85488b788b-pv49c"] Sep 10 14:42:56.355130 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.355060 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6jvfq"] Sep 10 14:42:56.355130 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.355073 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m5fkb"] Sep 10 14:42:56.355207 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.355136 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:56.357809 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.357791 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 10 14:42:56.357879 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.357828 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s4dwj\"" Sep 10 14:42:56.357879 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.357837 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 10 14:42:56.357879 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.357831 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 10 14:42:56.419395 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419363 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg9hw\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-kube-api-access-zg9hw\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419422 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5e5998f-51ba-4af7-8f00-334c457bd7fa-tmp-dir\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.419531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419468 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-installation-pull-secrets\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419490 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-bound-sa-token\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419507 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-registry-certificates\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419522 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e5998f-51ba-4af7-8f00-334c457bd7fa-config-volume\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.419791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419555 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.419791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419584 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-image-registry-private-configuration\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419655 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/934b4314-4204-4c57-bb87-4022d2719b88-ca-trust-extracted\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419688 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419712 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-trusted-ca\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.419791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.419741 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p2vd\" (UniqueName: \"kubernetes.io/projected/e5e5998f-51ba-4af7-8f00-334c457bd7fa-kube-api-access-5p2vd\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.520000 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.519935 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/934b4314-4204-4c57-bb87-4022d2719b88-ca-trust-extracted\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520000 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.519964 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520000 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.519980 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-trusted-ca\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520000 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.519998 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p2vd\" (UniqueName: \"kubernetes.io/projected/e5e5998f-51ba-4af7-8f00-334c457bd7fa-kube-api-access-5p2vd\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520024 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg9hw\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-kube-api-access-zg9hw\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520102 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520124 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85488b788b-pv49c: secret "image-registry-tls" not found Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520187 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls podName:934b4314-4204-4c57-bb87-4022d2719b88 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:57.020167277 +0000 UTC m=+33.692193032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls") pod "image-registry-85488b788b-pv49c" (UID: "934b4314-4204-4c57-bb87-4022d2719b88") : secret "image-registry-tls" not found Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5e5998f-51ba-4af7-8f00-334c457bd7fa-tmp-dir\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520275 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:56.520297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520288 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520319 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-installation-pull-secrets\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520343 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-bound-sa-token\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520387 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret podName:01b92fcc-fb48-4ed4-9bf8-86594f042671 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:28.520367206 +0000 UTC m=+65.192392968 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret") pod "global-pull-secret-syncer-zlhxp" (UID: "01b92fcc-fb48-4ed4-9bf8-86594f042671") : object "kube-system"/"original-pull-secret" not registered Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520400 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/934b4314-4204-4c57-bb87-4022d2719b88-ca-trust-extracted\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520416 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520470 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs podName:8a85e3e9-009d-4afd-9b07-235a6fb78af3 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:28.520455856 +0000 UTC m=+65.192481609 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs") pod "network-metrics-daemon-x27qd" (UID: "8a85e3e9-009d-4afd-9b07-235a6fb78af3") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520494 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-registry-certificates\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520529 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e5998f-51ba-4af7-8f00-334c457bd7fa-config-volume\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520569 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-image-registry-private-configuration\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520637 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b303f07a-2785-4c69-bd70-6ed299b78443-cert\") pod \"ingress-canary-6jvfq\" (UID: \"b303f07a-2785-4c69-bd70-6ed299b78443\") " pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520655 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5e5998f-51ba-4af7-8f00-334c457bd7fa-tmp-dir\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.520661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520663 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvx67\" (UniqueName: \"kubernetes.io/projected/b303f07a-2785-4c69-bd70-6ed299b78443-kube-api-access-mvx67\") pod \"ingress-canary-6jvfq\" (UID: \"b303f07a-2785-4c69-bd70-6ed299b78443\") " pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:56.521088 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520699 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 14:42:56.521088 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.520824 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls podName:e5e5998f-51ba-4af7-8f00-334c457bd7fa nodeName:}" failed. No retries permitted until 2026-09-10 14:42:57.020808264 +0000 UTC m=+33.692834042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls") pod "dns-default-m5fkb" (UID: "e5e5998f-51ba-4af7-8f00-334c457bd7fa") : secret "dns-default-metrics-tls" not found Sep 10 14:42:56.521088 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520956 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-registry-certificates\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.521088 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.520991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-trusted-ca\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.521088 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.521040 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e5998f-51ba-4af7-8f00-334c457bd7fa-config-volume\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.524221 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.524185 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-installation-pull-secrets\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.524221 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.524188 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-image-registry-private-configuration\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.529070 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.529052 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p2vd\" (UniqueName: \"kubernetes.io/projected/e5e5998f-51ba-4af7-8f00-334c457bd7fa-kube-api-access-5p2vd\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:56.529147 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.529098 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-bound-sa-token\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.529233 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.529217 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg9hw\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-kube-api-access-zg9hw\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:56.621704 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.621671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:56.621828 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.621810 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 10 14:42:56.621894 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.621831 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 10 14:42:56.621894 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.621841 2584 projected.go:196] Error preparing data for projected volume kube-api-access-nqgpg for pod openshift-network-diagnostics/network-check-target-njbnb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:56.621894 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.621839 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b303f07a-2785-4c69-bd70-6ed299b78443-cert\") pod \"ingress-canary-6jvfq\" (UID: \"b303f07a-2785-4c69-bd70-6ed299b78443\") " pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:56.621894 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.621879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvx67\" (UniqueName: \"kubernetes.io/projected/b303f07a-2785-4c69-bd70-6ed299b78443-kube-api-access-mvx67\") pod \"ingress-canary-6jvfq\" (UID: \"b303f07a-2785-4c69-bd70-6ed299b78443\") " pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:56.621894 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:56.621885 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg podName:9895ec05-d97a-4963-88db-96065abb41f4 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:28.621875142 +0000 UTC m=+65.293900893 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqgpg" (UniqueName: "kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg") pod "network-check-target-njbnb" (UID: "9895ec05-d97a-4963-88db-96065abb41f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 10 14:42:56.624031 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.624014 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b303f07a-2785-4c69-bd70-6ed299b78443-cert\") pod \"ingress-canary-6jvfq\" (UID: \"b303f07a-2785-4c69-bd70-6ed299b78443\") " pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:56.629525 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.629503 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvx67\" (UniqueName: \"kubernetes.io/projected/b303f07a-2785-4c69-bd70-6ed299b78443-kube-api-access-mvx67\") pod \"ingress-canary-6jvfq\" (UID: \"b303f07a-2785-4c69-bd70-6ed299b78443\") " pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:56.662960 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:56.662937 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6jvfq" Sep 10 14:42:57.024987 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.024899 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:57.024987 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.024961 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:57.025596 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:57.025073 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 14:42:57.025596 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:57.025139 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls podName:e5e5998f-51ba-4af7-8f00-334c457bd7fa nodeName:}" failed. No retries permitted until 2026-09-10 14:42:58.025124776 +0000 UTC m=+34.697150558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls") pod "dns-default-m5fkb" (UID: "e5e5998f-51ba-4af7-8f00-334c457bd7fa") : secret "dns-default-metrics-tls" not found Sep 10 14:42:57.025596 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:57.025074 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 14:42:57.025596 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:57.025174 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85488b788b-pv49c: secret "image-registry-tls" not found Sep 10 14:42:57.025596 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:57.025237 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls podName:934b4314-4204-4c57-bb87-4022d2719b88 nodeName:}" failed. No retries permitted until 2026-09-10 14:42:58.025221366 +0000 UTC m=+34.697247117 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls") pod "image-registry-85488b788b-pv49c" (UID: "934b4314-4204-4c57-bb87-4022d2719b88") : secret "image-registry-tls" not found Sep 10 14:42:57.756924 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.756888 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:42:57.757121 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.756902 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:42:57.757121 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.757032 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:42:57.759785 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.759766 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 14:42:57.759905 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.759829 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 14:42:57.759905 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.759867 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 14:42:57.761232 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.761213 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c8srj\"" Sep 10 14:42:57.761345 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.761248 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnf6m\"" Sep 10 14:42:57.761345 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:57.761257 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 14:42:58.032997 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:58.032916 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:42:58.032997 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:58.032981 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:42:58.033500 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:58.033044 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 14:42:58.033500 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:58.033120 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 14:42:58.033500 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:58.033128 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls podName:e5e5998f-51ba-4af7-8f00-334c457bd7fa nodeName:}" failed. No retries permitted until 2026-09-10 14:43:00.033104577 +0000 UTC m=+36.705130348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls") pod "dns-default-m5fkb" (UID: "e5e5998f-51ba-4af7-8f00-334c457bd7fa") : secret "dns-default-metrics-tls" not found Sep 10 14:42:58.033500 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:58.033133 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85488b788b-pv49c: secret "image-registry-tls" not found Sep 10 14:42:58.033500 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:42:58.033178 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls podName:934b4314-4204-4c57-bb87-4022d2719b88 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:00.033163196 +0000 UTC m=+36.705188955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls") pod "image-registry-85488b788b-pv49c" (UID: "934b4314-4204-4c57-bb87-4022d2719b88") : secret "image-registry-tls" not found Sep 10 14:42:58.562291 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:58.562263 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6jvfq"] Sep 10 14:42:58.566359 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:42:58.566335 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb303f07a_2785_4c69_bd70_6ed299b78443.slice/crio-b28ddf9b57a76b4acde4f26c7f5fbd9c2ea64adcd6748c2321956e33e977aea6 WatchSource:0}: Error finding container b28ddf9b57a76b4acde4f26c7f5fbd9c2ea64adcd6748c2321956e33e977aea6: Status 404 returned error can't find the container with id b28ddf9b57a76b4acde4f26c7f5fbd9c2ea64adcd6748c2321956e33e977aea6 Sep 10 14:42:59.139788 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:59.139554 2584 generic.go:356] "Generic (PLEG): container finished" podID="11e4848c-8666-4f5a-bc95-d0e5c8cc5154" containerID="1b9d37a305f6d0d62610c8bb73701d8a9690573f6543fa8ea55b89083dd1a5c0" exitCode=0 Sep 10 14:42:59.140445 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:59.139636 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerDied","Data":"1b9d37a305f6d0d62610c8bb73701d8a9690573f6543fa8ea55b89083dd1a5c0"} Sep 10 14:42:59.141315 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:42:59.141273 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6jvfq" event={"ID":"b303f07a-2785-4c69-bd70-6ed299b78443","Type":"ContainerStarted","Data":"b28ddf9b57a76b4acde4f26c7f5fbd9c2ea64adcd6748c2321956e33e977aea6"} Sep 10 14:43:00.047535 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:00.047500 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:00.047705 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:00.047548 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:43:00.047705 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:00.047642 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 14:43:00.047705 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:00.047644 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 14:43:00.047705 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:00.047704 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls podName:e5e5998f-51ba-4af7-8f00-334c457bd7fa nodeName:}" failed. No retries permitted until 2026-09-10 14:43:04.047690863 +0000 UTC m=+40.719716612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls") pod "dns-default-m5fkb" (UID: "e5e5998f-51ba-4af7-8f00-334c457bd7fa") : secret "dns-default-metrics-tls" not found Sep 10 14:43:00.047851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:00.047652 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85488b788b-pv49c: secret "image-registry-tls" not found Sep 10 14:43:00.047851 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:00.047731 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls podName:934b4314-4204-4c57-bb87-4022d2719b88 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:04.047725515 +0000 UTC m=+40.719751265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls") pod "image-registry-85488b788b-pv49c" (UID: "934b4314-4204-4c57-bb87-4022d2719b88") : secret "image-registry-tls" not found Sep 10 14:43:00.146645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:00.146611 2584 generic.go:356] "Generic (PLEG): container finished" podID="11e4848c-8666-4f5a-bc95-d0e5c8cc5154" containerID="f026d4437bbc144d2f7e1a47abda6b3e8de38f6e8cfce2c1f6270a967bd45283" exitCode=0 Sep 10 14:43:00.146645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:00.146650 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerDied","Data":"f026d4437bbc144d2f7e1a47abda6b3e8de38f6e8cfce2c1f6270a967bd45283"} Sep 10 14:43:01.151832 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:01.151801 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" event={"ID":"11e4848c-8666-4f5a-bc95-d0e5c8cc5154","Type":"ContainerStarted","Data":"8f01e948ead4303411bba9480651cdbf85ad4f7a77b99635e66a0d3721c6ce02"} Sep 10 14:43:01.174084 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:01.174038 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jfwfh" podStartSLOduration=7.785555181 podStartE2EDuration="38.17402607s" totalImagesPullingTime="30.388470889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:01.172692552 +0000 UTC m=+37.844718323" watchObservedRunningTime="2026-09-10 14:43:01.17402607 +0000 UTC m=+37.846051839" Sep 10 14:43:02.155521 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:02.155490 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6jvfq" event={"ID":"b303f07a-2785-4c69-bd70-6ed299b78443","Type":"ContainerStarted","Data":"085eb535c1a1933b09e29e1c476f27af68a16b00d87b2ec09855e1c61cdd6733"} Sep 10 14:43:02.173020 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:02.172979 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6jvfq" podStartSLOduration=3.193986423 podStartE2EDuration="6.172964434s" totalImagesPullingTime="2.978978011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:02.172507708 +0000 UTC m=+38.844533484" watchObservedRunningTime="2026-09-10 14:43:02.172964434 +0000 UTC m=+38.844990206" Sep 10 14:43:03.074463 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.074432 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zmw6z"] Sep 10 14:43:03.095750 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.095727 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zmw6z"] Sep 10 14:43:03.095888 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.095830 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.098512 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.098491 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 10 14:43:03.098624 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.098585 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 10 14:43:03.098624 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.098600 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fmx4m\"" Sep 10 14:43:03.098719 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.098626 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 10 14:43:03.099910 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.099892 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 10 14:43:03.272680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.272646 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f78edbc3-32a9-45ba-8160-cab3879b0ff8-crio-socket\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.273073 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.272696 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26vp6\" (UniqueName: \"kubernetes.io/projected/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-api-access-26vp6\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.273073 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.272760 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f78edbc3-32a9-45ba-8160-cab3879b0ff8-data-volume\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.273073 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.272830 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.273073 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.272867 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.373546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.373482 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f78edbc3-32a9-45ba-8160-cab3879b0ff8-crio-socket\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.373546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.373524 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26vp6\" (UniqueName: \"kubernetes.io/projected/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-api-access-26vp6\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.373546 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.373548 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f78edbc3-32a9-45ba-8160-cab3879b0ff8-data-volume\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.373742 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.373594 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.373742 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.373617 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.373742 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.373687 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 14:43:03.373742 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.373697 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f78edbc3-32a9-45ba-8160-cab3879b0ff8-crio-socket\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.373742 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.373722 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 14:43:03.373917 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.373754 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:03.873737317 +0000 UTC m=+40.545763089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : configmap "kube-rbac-proxy" not found Sep 10 14:43:03.373917 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.373807 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:03.873788459 +0000 UTC m=+40.545814229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : secret "insights-runtime-extractor-tls" not found Sep 10 14:43:03.373991 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.373926 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f78edbc3-32a9-45ba-8160-cab3879b0ff8-data-volume\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.389349 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.389326 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26vp6\" (UniqueName: \"kubernetes.io/projected/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-api-access-26vp6\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.394614 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.394592 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd"] Sep 10 14:43:03.411351 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.411329 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd"] Sep 10 14:43:03.411435 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.411428 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" Sep 10 14:43:03.414099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.414082 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nggcv\"" Sep 10 14:43:03.414099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.414090 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Sep 10 14:43:03.414099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.414100 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Sep 10 14:43:03.490290 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.490265 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-ljjnd"] Sep 10 14:43:03.505010 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.504987 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-ljjnd"] Sep 10 14:43:03.505108 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.505083 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:03.507727 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.507706 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 10 14:43:03.507915 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.507899 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-bt8lr\"" Sep 10 14:43:03.507979 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.507939 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 10 14:43:03.575613 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.575590 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rclkg\" (UniqueName: \"kubernetes.io/projected/eacd8d86-9eb3-48b8-a818-0b011faad98b-kube-api-access-rclkg\") pod \"migrator-fdbb8fdff-wrtfd\" (UID: \"eacd8d86-9eb3-48b8-a818-0b011faad98b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" Sep 10 14:43:03.676093 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.676032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:03.676093 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.676072 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rclkg\" (UniqueName: \"kubernetes.io/projected/eacd8d86-9eb3-48b8-a818-0b011faad98b-kube-api-access-rclkg\") pod \"migrator-fdbb8fdff-wrtfd\" (UID: \"eacd8d86-9eb3-48b8-a818-0b011faad98b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" Sep 10 14:43:03.676093 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.676091 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-nginx-conf\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:03.685077 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.685059 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rclkg\" (UniqueName: \"kubernetes.io/projected/eacd8d86-9eb3-48b8-a818-0b011faad98b-kube-api-access-rclkg\") pod \"migrator-fdbb8fdff-wrtfd\" (UID: \"eacd8d86-9eb3-48b8-a818-0b011faad98b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" Sep 10 14:43:03.719424 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.719404 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" Sep 10 14:43:03.776824 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.776712 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-nginx-conf\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:03.776958 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.776921 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:03.777040 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.777020 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 14:43:03.777407 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.777095 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert podName:ffce56c3-370a-44b9-a9b0-5d51b1fe63d2 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:04.27707496 +0000 UTC m=+40.949100726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ljjnd" (UID: "ffce56c3-370a-44b9-a9b0-5d51b1fe63d2") : secret "networking-console-plugin-cert" not found Sep 10 14:43:03.777693 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.777556 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-nginx-conf\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:03.877571 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.877538 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.877719 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:03.877589 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:03.877719 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.877683 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 14:43:03.877719 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.877697 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 14:43:03.877814 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.877744 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:04.877728181 +0000 UTC m=+41.549753932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : secret "insights-runtime-extractor-tls" not found Sep 10 14:43:03.877814 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:03.877759 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:04.877752302 +0000 UTC m=+41.549778051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : configmap "kube-rbac-proxy" not found Sep 10 14:43:04.000597 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.000571 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd"] Sep 10 14:43:04.003491 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:04.003469 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeacd8d86_9eb3_48b8_a818_0b011faad98b.slice/crio-1e09d4206b576a7772991b6faa63a6d834c2600486683281da61e9df8c251aa3 WatchSource:0}: Error finding container 1e09d4206b576a7772991b6faa63a6d834c2600486683281da61e9df8c251aa3: Status 404 returned error can't find the container with id 1e09d4206b576a7772991b6faa63a6d834c2600486683281da61e9df8c251aa3 Sep 10 14:43:04.079332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.079295 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:04.079512 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.079357 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:43:04.079512 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.079445 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 14:43:04.079512 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.079511 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls podName:e5e5998f-51ba-4af7-8f00-334c457bd7fa nodeName:}" failed. No retries permitted until 2026-09-10 14:43:12.079495592 +0000 UTC m=+48.751521354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls") pod "dns-default-m5fkb" (UID: "e5e5998f-51ba-4af7-8f00-334c457bd7fa") : secret "dns-default-metrics-tls" not found Sep 10 14:43:04.079624 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.079517 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 14:43:04.079624 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.079533 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85488b788b-pv49c: secret "image-registry-tls" not found Sep 10 14:43:04.079624 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.079594 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls podName:934b4314-4204-4c57-bb87-4022d2719b88 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:12.079579319 +0000 UTC m=+48.751605090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls") pod "image-registry-85488b788b-pv49c" (UID: "934b4314-4204-4c57-bb87-4022d2719b88") : secret "image-registry-tls" not found Sep 10 14:43:04.161231 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.161186 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" event={"ID":"eacd8d86-9eb3-48b8-a818-0b011faad98b","Type":"ContainerStarted","Data":"1e09d4206b576a7772991b6faa63a6d834c2600486683281da61e9df8c251aa3"} Sep 10 14:43:04.281313 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.281233 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:04.281683 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.281342 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 14:43:04.281683 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.281390 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert podName:ffce56c3-370a-44b9-a9b0-5d51b1fe63d2 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:05.281377291 +0000 UTC m=+41.953403040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ljjnd" (UID: "ffce56c3-370a-44b9-a9b0-5d51b1fe63d2") : secret "networking-console-plugin-cert" not found Sep 10 14:43:04.606563 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.606534 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qb7fw_416d5edd-4a50-4621-947f-46bffc5cb8c0/dns-node-resolver/0.log" Sep 10 14:43:04.890245 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.890143 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:04.890399 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.890293 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 14:43:04.890399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:04.890339 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:04.890399 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.890364 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:06.890344732 +0000 UTC m=+43.562370482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : configmap "kube-rbac-proxy" not found Sep 10 14:43:04.890565 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.890465 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 14:43:04.890565 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:04.890531 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:06.890512264 +0000 UTC m=+43.562538025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : secret "insights-runtime-extractor-tls" not found Sep 10 14:43:05.294382 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.294328 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:05.294745 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:05.294485 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 14:43:05.294745 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:05.294553 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert podName:ffce56c3-370a-44b9-a9b0-5d51b1fe63d2 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:07.294537453 +0000 UTC m=+43.966563208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ljjnd" (UID: "ffce56c3-370a-44b9-a9b0-5d51b1fe63d2") : secret "networking-console-plugin-cert" not found Sep 10 14:43:05.318819 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.318773 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-cqnx9"] Sep 10 14:43:05.332853 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.332824 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-cqnx9"] Sep 10 14:43:05.333006 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.332923 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.335832 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.335805 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-486q5\"" Sep 10 14:43:05.335944 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.335881 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 10 14:43:05.337157 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.337136 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 10 14:43:05.337157 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.337150 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 10 14:43:05.337274 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.337224 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 10 14:43:05.407177 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.407150 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6mpq9_cfcf1f64-077b-471c-8cba-0f0a90180acd/node-ca/0.log" Sep 10 14:43:05.496548 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.496509 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-signing-key\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.496671 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.496593 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrxdf\" (UniqueName: \"kubernetes.io/projected/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-kube-api-access-zrxdf\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.496726 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.496694 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-signing-cabundle\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.597355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.597273 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxdf\" (UniqueName: \"kubernetes.io/projected/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-kube-api-access-zrxdf\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.597510 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.597366 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-signing-cabundle\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.597510 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.597406 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-signing-key\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.598019 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.597987 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-signing-cabundle\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.599751 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.599731 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-signing-key\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.605712 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.605691 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrxdf\" (UniqueName: \"kubernetes.io/projected/f8ee7f52-d943-4084-ba03-84ad6fbdc99a-kube-api-access-zrxdf\") pod \"service-ca-75d99c484-cqnx9\" (UID: \"f8ee7f52-d943-4084-ba03-84ad6fbdc99a\") " pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.641876 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.641837 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-cqnx9" Sep 10 14:43:05.974377 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:05.974346 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-cqnx9"] Sep 10 14:43:05.977525 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:05.977496 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ee7f52_d943_4084_ba03_84ad6fbdc99a.slice/crio-5373c2599fdfb2290eec767e00a100633262dc9fccae29ea860c59b5bf61f724 WatchSource:0}: Error finding container 5373c2599fdfb2290eec767e00a100633262dc9fccae29ea860c59b5bf61f724: Status 404 returned error can't find the container with id 5373c2599fdfb2290eec767e00a100633262dc9fccae29ea860c59b5bf61f724 Sep 10 14:43:06.168857 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:06.168763 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" event={"ID":"eacd8d86-9eb3-48b8-a818-0b011faad98b","Type":"ContainerStarted","Data":"f2074af34d9b30be3139a0f0823afa1802f531d0a4f354651d100e5520d75586"} Sep 10 14:43:06.168857 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:06.168827 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" event={"ID":"eacd8d86-9eb3-48b8-a818-0b011faad98b","Type":"ContainerStarted","Data":"d75408ac4bfc23e54be9ffbc53f4e1150b24166ee3861cf655622151a9a46b58"} Sep 10 14:43:06.169930 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:06.169905 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-cqnx9" event={"ID":"f8ee7f52-d943-4084-ba03-84ad6fbdc99a","Type":"ContainerStarted","Data":"5373c2599fdfb2290eec767e00a100633262dc9fccae29ea860c59b5bf61f724"} Sep 10 14:43:06.188288 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:06.188237 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-wrtfd" podStartSLOduration=1.486638132 podStartE2EDuration="3.18822047s" totalImagesPullingTime="1.701582338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:06.187226281 +0000 UTC m=+42.859252051" watchObservedRunningTime="2026-09-10 14:43:06.18822047 +0000 UTC m=+42.860246236" Sep 10 14:43:06.207294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:06.207264 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6jvfq_b303f07a-2785-4c69-bd70-6ed299b78443/serve-healthcheck-canary/0.log" Sep 10 14:43:06.909303 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:06.909256 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:06.909852 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:06.909337 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:06.909852 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:06.909472 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 14:43:06.909852 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:06.909463 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 14:43:06.909852 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:06.909547 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:10.909532638 +0000 UTC m=+47.581558393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : configmap "kube-rbac-proxy" not found Sep 10 14:43:06.909852 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:06.909560 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:10.909554244 +0000 UTC m=+47.581579993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : secret "insights-runtime-extractor-tls" not found Sep 10 14:43:07.312134 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:07.312088 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:07.312379 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:07.312263 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 14:43:07.312379 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:07.312334 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert podName:ffce56c3-370a-44b9-a9b0-5d51b1fe63d2 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:11.312317151 +0000 UTC m=+47.984342901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ljjnd" (UID: "ffce56c3-370a-44b9-a9b0-5d51b1fe63d2") : secret "networking-console-plugin-cert" not found Sep 10 14:43:09.181634 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:09.181596 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-cqnx9" event={"ID":"f8ee7f52-d943-4084-ba03-84ad6fbdc99a","Type":"ContainerStarted","Data":"56d3e32dad079453b5248f11ba04612b76f177c4b6b65df6ca5dcaa0109259aa"} Sep 10 14:43:09.202945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:09.202888 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-cqnx9" podStartSLOduration=1.659280092 podStartE2EDuration="4.202870757s" totalImagesPullingTime="2.543590665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:09.20177022 +0000 UTC m=+45.873795994" watchObservedRunningTime="2026-09-10 14:43:09.202870757 +0000 UTC m=+45.874896559" Sep 10 14:43:10.944501 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:10.944461 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:10.944950 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:10.944513 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:10.944950 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:10.944628 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 10 14:43:10.944950 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:10.944636 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 14:43:10.944950 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:10.944682 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:18.94466716 +0000 UTC m=+55.616692923 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : configmap "kube-rbac-proxy" not found Sep 10 14:43:10.944950 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:10.944724 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:18.944705372 +0000 UTC m=+55.616731137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : secret "insights-runtime-extractor-tls" not found Sep 10 14:43:11.348685 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:11.348583 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:11.348847 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:11.348753 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 10 14:43:11.348847 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:11.348820 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert podName:ffce56c3-370a-44b9-a9b0-5d51b1fe63d2 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:19.348801372 +0000 UTC m=+56.020827137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-ljjnd" (UID: "ffce56c3-370a-44b9-a9b0-5d51b1fe63d2") : secret "networking-console-plugin-cert" not found Sep 10 14:43:12.155014 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:12.154958 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:12.155014 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:12.155018 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") pod \"image-registry-85488b788b-pv49c\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:43:12.155496 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:12.155142 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 10 14:43:12.155496 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:12.155165 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 10 14:43:12.155496 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:12.155178 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-85488b788b-pv49c: secret "image-registry-tls" not found Sep 10 14:43:12.155496 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:12.155243 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls podName:e5e5998f-51ba-4af7-8f00-334c457bd7fa nodeName:}" failed. No retries permitted until 2026-09-10 14:43:28.155221319 +0000 UTC m=+64.827247089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls") pod "dns-default-m5fkb" (UID: "e5e5998f-51ba-4af7-8f00-334c457bd7fa") : secret "dns-default-metrics-tls" not found Sep 10 14:43:12.155496 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:12.155265 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls podName:934b4314-4204-4c57-bb87-4022d2719b88 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:28.155255417 +0000 UTC m=+64.827281174 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls") pod "image-registry-85488b788b-pv49c" (UID: "934b4314-4204-4c57-bb87-4022d2719b88") : secret "image-registry-tls" not found Sep 10 14:43:19.010577 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:19.010543 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:19.011069 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:19.010591 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:19.011069 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:19.010685 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 10 14:43:19.011069 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:19.010747 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls podName:f78edbc3-32a9-45ba-8160-cab3879b0ff8 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:35.010731989 +0000 UTC m=+71.682757739 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zmw6z" (UID: "f78edbc3-32a9-45ba-8160-cab3879b0ff8") : secret "insights-runtime-extractor-tls" not found Sep 10 14:43:19.011069 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:19.011020 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f78edbc3-32a9-45ba-8160-cab3879b0ff8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:19.413188 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:19.413114 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:19.415440 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:19.415422 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ffce56c3-370a-44b9-a9b0-5d51b1fe63d2-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-ljjnd\" (UID: \"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2\") " pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:19.713059 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:19.713016 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" Sep 10 14:43:19.979674 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:19.979604 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-ljjnd"] Sep 10 14:43:19.982885 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:19.982861 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffce56c3_370a_44b9_a9b0_5d51b1fe63d2.slice/crio-95c028a9a954f2c3b451b5471434c5d8b263bab69d5ec68077f9a4a8733861b6 WatchSource:0}: Error finding container 95c028a9a954f2c3b451b5471434c5d8b263bab69d5ec68077f9a4a8733861b6: Status 404 returned error can't find the container with id 95c028a9a954f2c3b451b5471434c5d8b263bab69d5ec68077f9a4a8733861b6 Sep 10 14:43:20.211694 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:20.211654 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" event={"ID":"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2","Type":"ContainerStarted","Data":"95c028a9a954f2c3b451b5471434c5d8b263bab69d5ec68077f9a4a8733861b6"} Sep 10 14:43:21.216319 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:21.216288 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" event={"ID":"ffce56c3-370a-44b9-a9b0-5d51b1fe63d2","Type":"ContainerStarted","Data":"85f12d36a30fe8338d9350323b8204180f447a313d7ed94cee5219c6a5e3689e"} Sep 10 14:43:21.232101 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:21.232057 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-ljjnd" podStartSLOduration=17.303539466 podStartE2EDuration="18.232043901s" totalImagesPullingTime="928.504435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:21.231594096 +0000 UTC m=+57.903619869" watchObservedRunningTime="2026-09-10 14:43:21.232043901 +0000 UTC m=+57.904069672" Sep 10 14:43:23.132025 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:23.131992 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5z54x" Sep 10 14:43:25.118802 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.118772 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85488b788b-pv49c"] Sep 10 14:43:25.119147 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:25.118929 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-85488b788b-pv49c" podUID="934b4314-4204-4c57-bb87-4022d2719b88" Sep 10 14:43:25.122668 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.122644 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-lzdq5"] Sep 10 14:43:25.125277 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.125263 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm"] Sep 10 14:43:25.125403 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.125385 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-lzdq5" Sep 10 14:43:25.126787 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.126769 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" Sep 10 14:43:25.128237 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.128221 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-s89zr\"" Sep 10 14:43:25.128316 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.128290 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 10 14:43:25.128446 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.128434 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 10 14:43:25.129660 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.129643 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-q7nz7\"" Sep 10 14:43:25.129736 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.129660 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 10 14:43:25.139976 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.139951 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm"] Sep 10 14:43:25.140948 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.140930 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-lzdq5"] Sep 10 14:43:25.222934 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.222906 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-858d94df59-2fccl"] Sep 10 14:43:25.225495 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.225471 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.229505 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.229485 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:43:25.240049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.240033 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:43:25.242975 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.242954 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-858d94df59-2fccl"] Sep 10 14:43:25.258819 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.258794 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbnb8\" (UniqueName: \"kubernetes.io/projected/f0d8a303-d2bd-42e8-8b80-713a09fe9195-kube-api-access-jbnb8\") pod \"downloads-85d98c998f-lzdq5\" (UID: \"f0d8a303-d2bd-42e8-8b80-713a09fe9195\") " pod="openshift-console/downloads-85d98c998f-lzdq5" Sep 10 14:43:25.258917 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.258832 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-7r2qm\" (UID: \"3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" Sep 10 14:43:25.359751 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359723 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-image-registry-private-configuration\") pod \"934b4314-4204-4c57-bb87-4022d2719b88\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " Sep 10 14:43:25.359855 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359785 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg9hw\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-kube-api-access-zg9hw\") pod \"934b4314-4204-4c57-bb87-4022d2719b88\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " Sep 10 14:43:25.359855 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359803 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-bound-sa-token\") pod \"934b4314-4204-4c57-bb87-4022d2719b88\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " Sep 10 14:43:25.359855 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359825 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-installation-pull-secrets\") pod \"934b4314-4204-4c57-bb87-4022d2719b88\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " Sep 10 14:43:25.359855 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359843 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-registry-certificates\") pod \"934b4314-4204-4c57-bb87-4022d2719b88\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " Sep 10 14:43:25.360049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359859 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/934b4314-4204-4c57-bb87-4022d2719b88-ca-trust-extracted\") pod \"934b4314-4204-4c57-bb87-4022d2719b88\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " Sep 10 14:43:25.360049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359874 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-trusted-ca\") pod \"934b4314-4204-4c57-bb87-4022d2719b88\" (UID: \"934b4314-4204-4c57-bb87-4022d2719b88\") " Sep 10 14:43:25.360049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359939 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-7r2qm\" (UID: \"3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" Sep 10 14:43:25.360049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359969 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/943cf98b-80ce-43ba-bb4f-40ad7fcde845-ca-trust-extracted\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.359994 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whx9k\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-kube-api-access-whx9k\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360049 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360021 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-bound-sa-token\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360363 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/943cf98b-80ce-43ba-bb4f-40ad7fcde845-installation-pull-secrets\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360363 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360126 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/943cf98b-80ce-43ba-bb4f-40ad7fcde845-trusted-ca\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360363 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360169 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbnb8\" (UniqueName: \"kubernetes.io/projected/f0d8a303-d2bd-42e8-8b80-713a09fe9195-kube-api-access-jbnb8\") pod \"downloads-85d98c998f-lzdq5\" (UID: \"f0d8a303-d2bd-42e8-8b80-713a09fe9195\") " pod="openshift-console/downloads-85d98c998f-lzdq5" Sep 10 14:43:25.360363 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360264 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "934b4314-4204-4c57-bb87-4022d2719b88" (UID: "934b4314-4204-4c57-bb87-4022d2719b88"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:43:25.360363 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360268 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/943cf98b-80ce-43ba-bb4f-40ad7fcde845-image-registry-private-configuration\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360607 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360557 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "934b4314-4204-4c57-bb87-4022d2719b88" (UID: "934b4314-4204-4c57-bb87-4022d2719b88"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:43:25.360664 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360610 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-registry-tls\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360664 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360650 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/943cf98b-80ce-43ba-bb4f-40ad7fcde845-registry-certificates\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.360764 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360713 2584 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-registry-certificates\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:25.360764 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360729 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/934b4314-4204-4c57-bb87-4022d2719b88-trusted-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:25.360857 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.360823 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934b4314-4204-4c57-bb87-4022d2719b88-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "934b4314-4204-4c57-bb87-4022d2719b88" (UID: "934b4314-4204-4c57-bb87-4022d2719b88"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:43:25.362182 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.362140 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "934b4314-4204-4c57-bb87-4022d2719b88" (UID: "934b4314-4204-4c57-bb87-4022d2719b88"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:43:25.362288 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.362218 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-kube-api-access-zg9hw" (OuterVolumeSpecName: "kube-api-access-zg9hw") pod "934b4314-4204-4c57-bb87-4022d2719b88" (UID: "934b4314-4204-4c57-bb87-4022d2719b88"). InnerVolumeSpecName "kube-api-access-zg9hw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:43:25.362288 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.362236 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "934b4314-4204-4c57-bb87-4022d2719b88" (UID: "934b4314-4204-4c57-bb87-4022d2719b88"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:43:25.362361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.362309 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "934b4314-4204-4c57-bb87-4022d2719b88" (UID: "934b4314-4204-4c57-bb87-4022d2719b88"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:43:25.363066 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.363046 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-7r2qm\" (UID: \"3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" Sep 10 14:43:25.371151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.371103 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbnb8\" (UniqueName: \"kubernetes.io/projected/f0d8a303-d2bd-42e8-8b80-713a09fe9195-kube-api-access-jbnb8\") pod \"downloads-85d98c998f-lzdq5\" (UID: \"f0d8a303-d2bd-42e8-8b80-713a09fe9195\") " pod="openshift-console/downloads-85d98c998f-lzdq5" Sep 10 14:43:25.435480 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.435449 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-lzdq5" Sep 10 14:43:25.441854 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.441836 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" Sep 10 14:43:25.461976 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.461945 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/943cf98b-80ce-43ba-bb4f-40ad7fcde845-image-registry-private-configuration\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.461981 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-registry-tls\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462007 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/943cf98b-80ce-43ba-bb4f-40ad7fcde845-registry-certificates\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462040 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/943cf98b-80ce-43ba-bb4f-40ad7fcde845-ca-trust-extracted\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462057 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whx9k\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-kube-api-access-whx9k\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462076 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-bound-sa-token\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462137 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/943cf98b-80ce-43ba-bb4f-40ad7fcde845-installation-pull-secrets\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462167 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/943cf98b-80ce-43ba-bb4f-40ad7fcde845-trusted-ca\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.462399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462231 2584 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-bound-sa-token\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:25.462399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462242 2584 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-installation-pull-secrets\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:25.462399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462252 2584 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/934b4314-4204-4c57-bb87-4022d2719b88-ca-trust-extracted\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:25.462399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462261 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg9hw\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-kube-api-access-zg9hw\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:25.462399 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462274 2584 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/934b4314-4204-4c57-bb87-4022d2719b88-image-registry-private-configuration\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:25.463093 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.462799 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/943cf98b-80ce-43ba-bb4f-40ad7fcde845-ca-trust-extracted\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.463408 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.463351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/943cf98b-80ce-43ba-bb4f-40ad7fcde845-trusted-ca\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.463516 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.463430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/943cf98b-80ce-43ba-bb4f-40ad7fcde845-registry-certificates\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.464931 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.464892 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/943cf98b-80ce-43ba-bb4f-40ad7fcde845-image-registry-private-configuration\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.465434 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.465415 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-registry-tls\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.466050 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.466026 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/943cf98b-80ce-43ba-bb4f-40ad7fcde845-installation-pull-secrets\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.471299 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.471270 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-bound-sa-token\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.471568 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.471549 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whx9k\" (UniqueName: \"kubernetes.io/projected/943cf98b-80ce-43ba-bb4f-40ad7fcde845-kube-api-access-whx9k\") pod \"image-registry-858d94df59-2fccl\" (UID: \"943cf98b-80ce-43ba-bb4f-40ad7fcde845\") " pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.541582 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.541550 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gb7kw\"" Sep 10 14:43:25.548887 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.548872 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:25.712912 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.712887 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm"] Sep 10 14:43:25.715413 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:25.715384 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b23bc10_cb9e_4a5e_8454_4b2ce0c5cba2.slice/crio-e99852a383cf15be810409e93ba1117d5d81e6710205a7b0dbe6133adc5daa40 WatchSource:0}: Error finding container e99852a383cf15be810409e93ba1117d5d81e6710205a7b0dbe6133adc5daa40: Status 404 returned error can't find the container with id e99852a383cf15be810409e93ba1117d5d81e6710205a7b0dbe6133adc5daa40 Sep 10 14:43:25.715906 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.715885 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-lzdq5"] Sep 10 14:43:25.718961 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:25.718938 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d8a303_d2bd_42e8_8b80_713a09fe9195.slice/crio-3224f77b51f01779ceadd8de2c6de114c3789f3b1fbad8d560abd7a7363e41b3 WatchSource:0}: Error finding container 3224f77b51f01779ceadd8de2c6de114c3789f3b1fbad8d560abd7a7363e41b3: Status 404 returned error can't find the container with id 3224f77b51f01779ceadd8de2c6de114c3789f3b1fbad8d560abd7a7363e41b3 Sep 10 14:43:25.820010 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:25.819977 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-858d94df59-2fccl"] Sep 10 14:43:25.822449 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:25.822423 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943cf98b_80ce_43ba_bb4f_40ad7fcde845.slice/crio-aed3d2c864a184c3bff0af92e826a2f6de506fb7affe371abca929c95ba4536d WatchSource:0}: Error finding container aed3d2c864a184c3bff0af92e826a2f6de506fb7affe371abca929c95ba4536d: Status 404 returned error can't find the container with id aed3d2c864a184c3bff0af92e826a2f6de506fb7affe371abca929c95ba4536d Sep 10 14:43:26.236408 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.236366 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-858d94df59-2fccl" event={"ID":"943cf98b-80ce-43ba-bb4f-40ad7fcde845","Type":"ContainerStarted","Data":"28553f508d7d84e36f619d40b0bf61085b6ede56af0fc21a4dd04acd4b170941"} Sep 10 14:43:26.236846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.236429 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-858d94df59-2fccl" event={"ID":"943cf98b-80ce-43ba-bb4f-40ad7fcde845","Type":"ContainerStarted","Data":"aed3d2c864a184c3bff0af92e826a2f6de506fb7affe371abca929c95ba4536d"} Sep 10 14:43:26.236846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.236564 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:26.237849 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.237824 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-lzdq5" event={"ID":"f0d8a303-d2bd-42e8-8b80-713a09fe9195","Type":"ContainerStarted","Data":"3224f77b51f01779ceadd8de2c6de114c3789f3b1fbad8d560abd7a7363e41b3"} Sep 10 14:43:26.239218 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.239186 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85488b788b-pv49c" Sep 10 14:43:26.239326 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.239222 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" event={"ID":"3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2","Type":"ContainerStarted","Data":"e99852a383cf15be810409e93ba1117d5d81e6710205a7b0dbe6133adc5daa40"} Sep 10 14:43:26.258936 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.258603 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-858d94df59-2fccl" podStartSLOduration=1.258565409 podStartE2EDuration="1.258565409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:26.257436216 +0000 UTC m=+62.929461989" watchObservedRunningTime="2026-09-10 14:43:26.258565409 +0000 UTC m=+62.930591183" Sep 10 14:43:26.299688 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.299652 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-85488b788b-pv49c"] Sep 10 14:43:26.301687 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.301665 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-85488b788b-pv49c"] Sep 10 14:43:26.370517 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:26.370269 2584 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/934b4314-4204-4c57-bb87-4022d2719b88-registry-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:43:27.243941 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:27.243900 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" event={"ID":"3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2","Type":"ContainerStarted","Data":"0d352f3c3134e75f4e8191e697b216b53c17561f6abb4dd4f57349f33f39167f"} Sep 10 14:43:27.244423 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:27.244295 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" Sep 10 14:43:27.250709 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:27.250683 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" Sep 10 14:43:27.260680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:27.260628 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-7r2qm" podStartSLOduration=1.329636684 podStartE2EDuration="2.260611902s" totalImagesPullingTime="930.975218ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:27.25931886 +0000 UTC m=+63.931344631" watchObservedRunningTime="2026-09-10 14:43:27.260611902 +0000 UTC m=+63.932637674" Sep 10 14:43:27.761269 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:27.761235 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="934b4314-4204-4c57-bb87-4022d2719b88" path="/var/lib/kubelet/pods/934b4314-4204-4c57-bb87-4022d2719b88/volumes" Sep 10 14:43:28.187163 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.187079 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:28.189641 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.189614 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e5e5998f-51ba-4af7-8f00-334c457bd7fa-metrics-tls\") pod \"dns-default-m5fkb\" (UID: \"e5e5998f-51ba-4af7-8f00-334c457bd7fa\") " pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:28.446983 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.446949 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mkqg5\"" Sep 10 14:43:28.454316 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.454285 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:28.591058 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.591026 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:43:28.591230 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.591093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:43:28.594538 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.594486 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 10 14:43:28.594538 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.594515 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 10 14:43:28.604463 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.604438 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01b92fcc-fb48-4ed4-9bf8-86594f042671-original-pull-secret\") pod \"global-pull-secret-syncer-zlhxp\" (UID: \"01b92fcc-fb48-4ed4-9bf8-86594f042671\") " pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:43:28.604852 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.604833 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a85e3e9-009d-4afd-9b07-235a6fb78af3-metrics-certs\") pod \"network-metrics-daemon-x27qd\" (UID: \"8a85e3e9-009d-4afd-9b07-235a6fb78af3\") " pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:43:28.676776 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.676746 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zlhxp" Sep 10 14:43:28.685664 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.685638 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnf6m\"" Sep 10 14:43:28.691409 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.691387 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:43:28.693339 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.693319 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x27qd" Sep 10 14:43:28.694428 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.694413 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 10 14:43:28.704465 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.704413 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 10 14:43:28.715328 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.715307 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqgpg\" (UniqueName: \"kubernetes.io/projected/9895ec05-d97a-4963-88db-96065abb41f4-kube-api-access-nqgpg\") pod \"network-check-target-njbnb\" (UID: \"9895ec05-d97a-4963-88db-96065abb41f4\") " pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:43:28.744700 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.744626 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m5fkb"] Sep 10 14:43:28.747309 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:28.747277 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5998f_51ba_4af7_8f00_334c457bd7fa.slice/crio-f72e84497c6a144c60029dd639d74a796ac2bdf0235261faeaa425566ee01ace WatchSource:0}: Error finding container f72e84497c6a144c60029dd639d74a796ac2bdf0235261faeaa425566ee01ace: Status 404 returned error can't find the container with id f72e84497c6a144c60029dd639d74a796ac2bdf0235261faeaa425566ee01ace Sep 10 14:43:28.975315 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.975239 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c8srj\"" Sep 10 14:43:28.980795 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.980768 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zlhxp"] Sep 10 14:43:28.982685 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:28.982665 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:43:28.983658 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:28.983635 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b92fcc_fb48_4ed4_9bf8_86594f042671.slice/crio-2f5df00a35f6d9aa59ccdaf87c39509371f63ae0baf359efb026dd33248c9176 WatchSource:0}: Error finding container 2f5df00a35f6d9aa59ccdaf87c39509371f63ae0baf359efb026dd33248c9176: Status 404 returned error can't find the container with id 2f5df00a35f6d9aa59ccdaf87c39509371f63ae0baf359efb026dd33248c9176 Sep 10 14:43:29.004892 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:29.004868 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x27qd"] Sep 10 14:43:29.007906 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:29.007882 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a85e3e9_009d_4afd_9b07_235a6fb78af3.slice/crio-57794953267b0d0babb82cb4315fb2424e3fabf1236d564a41fdd97c1d9012e0 WatchSource:0}: Error finding container 57794953267b0d0babb82cb4315fb2424e3fabf1236d564a41fdd97c1d9012e0: Status 404 returned error can't find the container with id 57794953267b0d0babb82cb4315fb2424e3fabf1236d564a41fdd97c1d9012e0 Sep 10 14:43:29.253082 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:29.252995 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x27qd" event={"ID":"8a85e3e9-009d-4afd-9b07-235a6fb78af3","Type":"ContainerStarted","Data":"57794953267b0d0babb82cb4315fb2424e3fabf1236d564a41fdd97c1d9012e0"} Sep 10 14:43:29.254596 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:29.254526 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m5fkb" event={"ID":"e5e5998f-51ba-4af7-8f00-334c457bd7fa","Type":"ContainerStarted","Data":"f72e84497c6a144c60029dd639d74a796ac2bdf0235261faeaa425566ee01ace"} Sep 10 14:43:29.255705 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:29.255681 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zlhxp" event={"ID":"01b92fcc-fb48-4ed4-9bf8-86594f042671","Type":"ContainerStarted","Data":"2f5df00a35f6d9aa59ccdaf87c39509371f63ae0baf359efb026dd33248c9176"} Sep 10 14:43:29.285971 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:29.285942 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-njbnb"] Sep 10 14:43:29.289658 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:29.289621 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9895ec05_d97a_4963_88db_96065abb41f4.slice/crio-21d24630c68b63f51a3e2fca8cb97ef68e9d590a3e19407da4852424fc45c26e WatchSource:0}: Error finding container 21d24630c68b63f51a3e2fca8cb97ef68e9d590a3e19407da4852424fc45c26e: Status 404 returned error can't find the container with id 21d24630c68b63f51a3e2fca8cb97ef68e9d590a3e19407da4852424fc45c26e Sep 10 14:43:30.265505 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:30.265416 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-njbnb" event={"ID":"9895ec05-d97a-4963-88db-96065abb41f4","Type":"ContainerStarted","Data":"21d24630c68b63f51a3e2fca8cb97ef68e9d590a3e19407da4852424fc45c26e"} Sep 10 14:43:31.274195 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:31.273903 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x27qd" event={"ID":"8a85e3e9-009d-4afd-9b07-235a6fb78af3","Type":"ContainerStarted","Data":"341c1a57741ec7794301af6523732546a0895e9ec62bee3a0916a366d92d442a"} Sep 10 14:43:31.274195 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:31.274048 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x27qd" event={"ID":"8a85e3e9-009d-4afd-9b07-235a6fb78af3","Type":"ContainerStarted","Data":"815e2f2210aed26e27d07e00eba00e84320aabb65f9ba14f81aa2cb70a923aa4"} Sep 10 14:43:31.310183 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:31.310117 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x27qd" podStartSLOduration=66.794921373 podStartE2EDuration="1m8.310097044s" totalImagesPullingTime="1.515175671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:31.309605082 +0000 UTC m=+67.981630857" watchObservedRunningTime="2026-09-10 14:43:31.310097044 +0000 UTC m=+67.982122818" Sep 10 14:43:32.281001 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.280967 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m5fkb" event={"ID":"e5e5998f-51ba-4af7-8f00-334c457bd7fa","Type":"ContainerStarted","Data":"17d6640c73e5a501da9350b1946fb027854a23cdacedeaf2f3f448f2a94d2efd"} Sep 10 14:43:32.281454 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.281021 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m5fkb" event={"ID":"e5e5998f-51ba-4af7-8f00-334c457bd7fa","Type":"ContainerStarted","Data":"87be42d4a18bf2568d5b85a6ded359351fba3bc081bd97fe8f874dc5820976e0"} Sep 10 14:43:32.281454 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.281246 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:32.302945 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.302585 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m5fkb" podStartSLOduration=33.472939244 podStartE2EDuration="36.302565729s" totalImagesPullingTime="2.829626485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:32.301158843 +0000 UTC m=+68.973184620" watchObservedRunningTime="2026-09-10 14:43:32.302565729 +0000 UTC m=+68.974591503" Sep 10 14:43:32.894982 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.894944 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l652x"] Sep 10 14:43:32.898663 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.897434 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:32.902647 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.901633 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 10 14:43:32.902647 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.901903 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 10 14:43:32.902647 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.902092 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 10 14:43:32.902647 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.902284 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vfxxz\"" Sep 10 14:43:32.907390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.904760 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 10 14:43:32.907390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.904971 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 10 14:43:32.907390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:32.905154 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.030954 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-accelerators-collector-config\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031001 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-tls\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031029 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031073 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-root\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031100 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/623acfa3-a4c4-415a-ad9f-13c298fd6b42-metrics-client-ca\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031129 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shh2z\" (UniqueName: \"kubernetes.io/projected/623acfa3-a4c4-415a-ad9f-13c298fd6b42-kube-api-access-shh2z\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031158 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-sys\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031181 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-textfile\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.031219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.031230 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-wtmp\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131727 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-wtmp\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131797 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-accelerators-collector-config\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131824 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-tls\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131849 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131892 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-root\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131919 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/623acfa3-a4c4-415a-ad9f-13c298fd6b42-metrics-client-ca\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131949 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shh2z\" (UniqueName: \"kubernetes.io/projected/623acfa3-a4c4-415a-ad9f-13c298fd6b42-kube-api-access-shh2z\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.131981 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-sys\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.132744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.132003 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-textfile\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.133309 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.133007 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-textfile\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.133309 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.133091 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-root\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.134105 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.133490 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/623acfa3-a4c4-415a-ad9f-13c298fd6b42-metrics-client-ca\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.134105 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.133745 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-sys\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.134105 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:33.133810 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 10 14:43:33.134105 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:43:33.133848 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-tls podName:623acfa3-a4c4-415a-ad9f-13c298fd6b42 nodeName:}" failed. No retries permitted until 2026-09-10 14:43:33.633835695 +0000 UTC m=+70.305861445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-tls") pod "node-exporter-l652x" (UID: "623acfa3-a4c4-415a-ad9f-13c298fd6b42") : secret "node-exporter-tls" not found Sep 10 14:43:33.134105 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.133859 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-accelerators-collector-config\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.134105 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.133980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-wtmp\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.143217 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.143142 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.152138 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.152074 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shh2z\" (UniqueName: \"kubernetes.io/projected/623acfa3-a4c4-415a-ad9f-13c298fd6b42-kube-api-access-shh2z\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.637103 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.637065 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-tls\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.639636 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.639612 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/623acfa3-a4c4-415a-ad9f-13c298fd6b42-node-exporter-tls\") pod \"node-exporter-l652x\" (UID: \"623acfa3-a4c4-415a-ad9f-13c298fd6b42\") " pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:33.815756 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:33.815728 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l652x" Sep 10 14:43:35.048793 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:35.048752 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:35.051853 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:35.051827 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f78edbc3-32a9-45ba-8160-cab3879b0ff8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zmw6z\" (UID: \"f78edbc3-32a9-45ba-8160-cab3879b0ff8\") " pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:35.207166 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:35.207129 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fmx4m\"" Sep 10 14:43:35.215192 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:35.215158 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zmw6z" Sep 10 14:43:35.348407 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:35.348373 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623acfa3_a4c4_415a_ad9f_13c298fd6b42.slice/crio-a797e5b492cc8474cb61b300ee8b923c2c6a8747dcfd74c3ef3ea445896b7a15 WatchSource:0}: Error finding container a797e5b492cc8474cb61b300ee8b923c2c6a8747dcfd74c3ef3ea445896b7a15: Status 404 returned error can't find the container with id a797e5b492cc8474cb61b300ee8b923c2c6a8747dcfd74c3ef3ea445896b7a15 Sep 10 14:43:35.618533 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:35.618454 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zmw6z"] Sep 10 14:43:35.621656 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:35.621625 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78edbc3_32a9_45ba_8160_cab3879b0ff8.slice/crio-afdaaa2f3c8ab37dc13d78a05b4f67ea9dca2405da5a4fd5e134f103ad485cb7 WatchSource:0}: Error finding container afdaaa2f3c8ab37dc13d78a05b4f67ea9dca2405da5a4fd5e134f103ad485cb7: Status 404 returned error can't find the container with id afdaaa2f3c8ab37dc13d78a05b4f67ea9dca2405da5a4fd5e134f103ad485cb7 Sep 10 14:43:36.077369 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.077332 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b945f867d-lvrvx"] Sep 10 14:43:36.090391 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.090361 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.092045 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.092019 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b945f867d-lvrvx"] Sep 10 14:43:36.093897 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.093596 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2vndb\"" Sep 10 14:43:36.093897 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.093743 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 10 14:43:36.094070 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.093976 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 10 14:43:36.094161 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.094147 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 10 14:43:36.095052 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.094904 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 10 14:43:36.095506 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.095477 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 10 14:43:36.100379 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.100357 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 10 14:43:36.156647 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.156494 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5fmx\" (UniqueName: \"kubernetes.io/projected/3d191cb0-4c25-4468-b463-b9b816bfd199-kube-api-access-v5fmx\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.156647 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.156571 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-service-ca\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.156647 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.156602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-console-config\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.156914 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.156655 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-oauth-serving-cert\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.156914 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.156692 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-oauth-config\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.156914 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.156717 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-serving-cert\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.156914 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.156746 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-trusted-ca-bundle\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.258355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.257836 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-oauth-config\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.258355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.257888 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-serving-cert\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.258355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.258036 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-trusted-ca-bundle\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.258355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.258069 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5fmx\" (UniqueName: \"kubernetes.io/projected/3d191cb0-4c25-4468-b463-b9b816bfd199-kube-api-access-v5fmx\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.258355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.258136 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-service-ca\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.258355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.258172 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-console-config\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.258355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.258226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-oauth-serving-cert\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.259667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.258947 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-console-config\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.259667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.259239 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-trusted-ca-bundle\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.259667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.259611 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-oauth-serving-cert\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.259667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.259639 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-service-ca\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.261263 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.261067 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-oauth-config\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.261263 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.261218 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-serving-cert\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.280143 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.280118 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5fmx\" (UniqueName: \"kubernetes.io/projected/3d191cb0-4c25-4468-b463-b9b816bfd199-kube-api-access-v5fmx\") pod \"console-7b945f867d-lvrvx\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.300837 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.300802 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l652x" event={"ID":"623acfa3-a4c4-415a-ad9f-13c298fd6b42","Type":"ContainerStarted","Data":"a797e5b492cc8474cb61b300ee8b923c2c6a8747dcfd74c3ef3ea445896b7a15"} Sep 10 14:43:36.303243 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.303212 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zlhxp" event={"ID":"01b92fcc-fb48-4ed4-9bf8-86594f042671","Type":"ContainerStarted","Data":"4abe355c72cd3fee1a66972acbae50783eb171bb0600b3db2012f7ff5925f64b"} Sep 10 14:43:36.304897 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.304876 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-njbnb" event={"ID":"9895ec05-d97a-4963-88db-96065abb41f4","Type":"ContainerStarted","Data":"4eef92b8857f6ad951a8de1fab31fc3792cf44bf91f8c44369bf81b706a8fd20"} Sep 10 14:43:36.305263 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.305244 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:43:36.308075 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.307495 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zmw6z" event={"ID":"f78edbc3-32a9-45ba-8160-cab3879b0ff8","Type":"ContainerStarted","Data":"d4c9da27fe3595e38a81e895e9c9232768619ce79c58085617ea6743bdf9da74"} Sep 10 14:43:36.308075 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.307540 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zmw6z" event={"ID":"f78edbc3-32a9-45ba-8160-cab3879b0ff8","Type":"ContainerStarted","Data":"afdaaa2f3c8ab37dc13d78a05b4f67ea9dca2405da5a4fd5e134f103ad485cb7"} Sep 10 14:43:36.320374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.320327 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zlhxp" podStartSLOduration=66.942050892 podStartE2EDuration="1m13.320310175s" totalImagesPullingTime="6.378259283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:36.31876195 +0000 UTC m=+72.990787726" watchObservedRunningTime="2026-09-10 14:43:36.320310175 +0000 UTC m=+72.992335948" Sep 10 14:43:36.406645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.406568 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:36.746092 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.746032 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-njbnb" podStartSLOduration=67.688587747 podStartE2EDuration="1m13.746010755s" totalImagesPullingTime="6.057423008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:36.33496078 +0000 UTC m=+73.006986553" watchObservedRunningTime="2026-09-10 14:43:36.746010755 +0000 UTC m=+73.418036528" Sep 10 14:43:36.747243 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:36.747186 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b945f867d-lvrvx"] Sep 10 14:43:36.753257 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:36.753229 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d191cb0_4c25_4468_b463_b9b816bfd199.slice/crio-f8c278ee78dff5c2dd6ef4f18fcdfaf7be1b7a58c538580198e91e2c056e35f6 WatchSource:0}: Error finding container f8c278ee78dff5c2dd6ef4f18fcdfaf7be1b7a58c538580198e91e2c056e35f6: Status 404 returned error can't find the container with id f8c278ee78dff5c2dd6ef4f18fcdfaf7be1b7a58c538580198e91e2c056e35f6 Sep 10 14:43:37.313771 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:37.313732 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zmw6z" event={"ID":"f78edbc3-32a9-45ba-8160-cab3879b0ff8","Type":"ContainerStarted","Data":"6a1de8087662a93b1d20000ecc19aff3668af70150d89bc066a8ca313e4d0276"} Sep 10 14:43:37.315149 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:37.315113 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b945f867d-lvrvx" event={"ID":"3d191cb0-4c25-4468-b463-b9b816bfd199","Type":"ContainerStarted","Data":"f8c278ee78dff5c2dd6ef4f18fcdfaf7be1b7a58c538580198e91e2c056e35f6"} Sep 10 14:43:37.316952 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:37.316921 2584 generic.go:356] "Generic (PLEG): container finished" podID="623acfa3-a4c4-415a-ad9f-13c298fd6b42" containerID="a0158d5892d48e58ff437afeb9144743ce90dcc100e71517a9fccf9b6a0f5e43" exitCode=0 Sep 10 14:43:37.317064 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:37.317016 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l652x" event={"ID":"623acfa3-a4c4-415a-ad9f-13c298fd6b42","Type":"ContainerDied","Data":"a0158d5892d48e58ff437afeb9144743ce90dcc100e71517a9fccf9b6a0f5e43"} Sep 10 14:43:39.108112 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.108073 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:43:39.111160 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.111140 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.114032 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.113969 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 14:43:39.114032 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.113971 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 14:43:39.114032 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.114013 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 14:43:39.114544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.114343 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xj28m\"" Sep 10 14:43:39.114544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.114370 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 14:43:39.114544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.114385 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 14:43:39.114544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.114406 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 14:43:39.114544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.114421 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 14:43:39.114544 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.114446 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 14:43:39.115643 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.115500 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 14:43:39.115643 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.115550 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1f8cfsfndgdv3\"" Sep 10 14:43:39.115643 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.115567 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 14:43:39.115843 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.115644 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 14:43:39.116771 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.116748 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 14:43:39.116999 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.116977 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 14:43:39.117090 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.116982 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 14:43:39.118576 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.118557 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 14:43:39.136951 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.136928 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:43:39.185551 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185455 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-config-out\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185551 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-web-config\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185551 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185528 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185559 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7dzd\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-kube-api-access-k7dzd\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185631 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185670 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185706 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185733 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185762 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185791 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.185827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185827 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185870 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185900 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185923 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185954 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.185982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.186011 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-config\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.186051 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.186079 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.186159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.186105 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.286815 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.286779 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.286996 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.286833 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.286996 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.286868 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287125 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.286977 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287125 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287102 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287226 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287138 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287226 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287166 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-config\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287225 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287253 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287285 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287326 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-config-out\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287348 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-web-config\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287373 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287396 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7dzd\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-kube-api-access-k7dzd\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.287673 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287525 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287869 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.287869 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288284 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288392 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288733 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288767 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288807 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288837 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288868 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.289294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.288898 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.291361 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.291372 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.291790 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.291891 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.292033 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-config\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.292315 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292699 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.292476 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.292817 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.292792 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.293454 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.293434 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.293663 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.293640 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.294031 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.293995 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-config-out\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.294332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.294312 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.294447 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.294424 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-web-config\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.294911 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.294888 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.296471 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.296453 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7dzd\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-kube-api-access-k7dzd\") pod \"prometheus-k8s-0\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:39.422716 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:39.422626 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:43:42.287234 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:42.287183 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m5fkb" Sep 10 14:43:44.044401 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.044363 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:43:44.047179 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:43:44.046904 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b631b22_06ef_421f_87cf_a395b0849887.slice/crio-68cef6dcc22fd6c682aae8e10ac29527bd72c7eb19079e27c3e9fca39cc47edc WatchSource:0}: Error finding container 68cef6dcc22fd6c682aae8e10ac29527bd72c7eb19079e27c3e9fca39cc47edc: Status 404 returned error can't find the container with id 68cef6dcc22fd6c682aae8e10ac29527bd72c7eb19079e27c3e9fca39cc47edc Sep 10 14:43:44.357068 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.356816 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l652x" event={"ID":"623acfa3-a4c4-415a-ad9f-13c298fd6b42","Type":"ContainerStarted","Data":"4c02ac7d2adf87e832614c777b1f3f8646c0842e092fe3fec24e066ab167664f"} Sep 10 14:43:44.357068 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.356927 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l652x" event={"ID":"623acfa3-a4c4-415a-ad9f-13c298fd6b42","Type":"ContainerStarted","Data":"0fd2c6844fd244289252ed88044d9ef6b09656bfc23ef8357912dcadd9592cbf"} Sep 10 14:43:44.359789 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.359761 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerStarted","Data":"68cef6dcc22fd6c682aae8e10ac29527bd72c7eb19079e27c3e9fca39cc47edc"} Sep 10 14:43:44.363099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.362958 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-lzdq5" event={"ID":"f0d8a303-d2bd-42e8-8b80-713a09fe9195","Type":"ContainerStarted","Data":"502a0cb89091be7c584a14bc65f7aa73657c63fa0fd7cd00f1825febd41febf2"} Sep 10 14:43:44.364058 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.363718 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-lzdq5" Sep 10 14:43:44.377559 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.377529 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-lzdq5" Sep 10 14:43:44.379937 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.379720 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l652x" podStartSLOduration=11.187656158 podStartE2EDuration="12.379703265s" totalImagesPullingTime="1.119671387s" totalInitContainerRuntime="72.37572ms" isStatefulPod=true podCreationTimestamp="2026-09-10 14:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:44.378348772 +0000 UTC m=+81.050374540" watchObservedRunningTime="2026-09-10 14:43:44.379703265 +0000 UTC m=+81.051729041" Sep 10 14:43:44.396644 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:44.395608 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-lzdq5" podStartSLOduration=1.245406999 podStartE2EDuration="19.395589316s" totalImagesPullingTime="18.150182317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:44.394691779 +0000 UTC m=+81.066717554" watchObservedRunningTime="2026-09-10 14:43:44.395589316 +0000 UTC m=+81.067615092" Sep 10 14:43:47.251146 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:47.251021 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-858d94df59-2fccl" Sep 10 14:43:48.383224 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:48.383176 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b945f867d-lvrvx" event={"ID":"3d191cb0-4c25-4468-b463-b9b816bfd199","Type":"ContainerStarted","Data":"ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a"} Sep 10 14:43:48.385070 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:48.385045 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b631b22-06ef-421f-87cf-a395b0849887" containerID="c4fb097f941352971e3039b906a049075f4f534cc5f47e154f25a9997cc8c357" exitCode=0 Sep 10 14:43:48.385192 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:48.385129 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"c4fb097f941352971e3039b906a049075f4f534cc5f47e154f25a9997cc8c357"} Sep 10 14:43:48.393637 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:48.393612 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zmw6z" event={"ID":"f78edbc3-32a9-45ba-8160-cab3879b0ff8","Type":"ContainerStarted","Data":"132439c386ecef477367228b08a70849b4698acb5e4d64edfa7c6537704fa781"} Sep 10 14:43:48.406132 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:48.406065 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b945f867d-lvrvx" podStartSLOduration=1.291057761 podStartE2EDuration="12.406052511s" totalImagesPullingTime="11.11499475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:48.405397302 +0000 UTC m=+85.077423075" watchObservedRunningTime="2026-09-10 14:43:48.406052511 +0000 UTC m=+85.078078282" Sep 10 14:43:48.447323 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:48.447282 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zmw6z" podStartSLOduration=33.356354035 podStartE2EDuration="45.447269306s" totalImagesPullingTime="12.090915271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:43:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:48.445747624 +0000 UTC m=+85.117773396" watchObservedRunningTime="2026-09-10 14:43:48.447269306 +0000 UTC m=+85.119295077" Sep 10 14:43:53.420487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:53.420452 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerStarted","Data":"41fba61e646e0a1e2cb8b4704e8df9cc98b42561fb59fa11502bec3a08434deb"} Sep 10 14:43:53.421062 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:53.420548 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerStarted","Data":"e198fe4b16bbfa1f6e62307782d6211594a80bbbc8f28f0e8c62bd45ea6f135d"} Sep 10 14:43:54.597276 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:54.597243 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b945f867d-lvrvx"] Sep 10 14:43:55.431617 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:55.431588 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerStarted","Data":"4aac8073f77602622390185b7d687cf81cb9925a17672256397d1a51691a119a"} Sep 10 14:43:55.431738 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:55.431653 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerStarted","Data":"79c79e16c0183ec5001542943ad6446b4558400ef791fce2f695fe99777cf780"} Sep 10 14:43:55.431738 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:55.431683 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerStarted","Data":"9c2363b4beaed0e785d3945144f1902f181eb0edea6e66dfaacb0f67cc92d807"} Sep 10 14:43:56.407567 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:56.407521 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:43:56.438712 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:56.438680 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerStarted","Data":"1cf8c73a224a34f0621b8b681becd2943c1bf5565790ec11c059fc4f05fb8b99"} Sep 10 14:43:56.467850 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:56.467803 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.162939494 podStartE2EDuration="17.467787333s" totalImagesPullingTime="10.274648761s" totalInitContainerRuntime="30.199078ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:56.465152228 +0000 UTC m=+93.137178000" watchObservedRunningTime="2026-09-10 14:43:56.467787333 +0000 UTC m=+93.139813104" Sep 10 14:43:59.423388 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:43:59.423351 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:08.323343 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:08.323309 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-njbnb" Sep 10 14:44:19.619435 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:19.619384 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b945f867d-lvrvx" podUID="3d191cb0-4c25-4468-b463-b9b816bfd199" containerName="console" containerID="cri-o://ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a" gracePeriod=15 Sep 10 14:44:19.935139 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:19.935100 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:44:20.022557 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.022519 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-oauth-config\") pod \"3d191cb0-4c25-4468-b463-b9b816bfd199\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " Sep 10 14:44:20.022731 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.022599 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5fmx\" (UniqueName: \"kubernetes.io/projected/3d191cb0-4c25-4468-b463-b9b816bfd199-kube-api-access-v5fmx\") pod \"3d191cb0-4c25-4468-b463-b9b816bfd199\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " Sep 10 14:44:20.022731 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.022635 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-trusted-ca-bundle\") pod \"3d191cb0-4c25-4468-b463-b9b816bfd199\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " Sep 10 14:44:20.022731 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.022664 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-serving-cert\") pod \"3d191cb0-4c25-4468-b463-b9b816bfd199\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " Sep 10 14:44:20.022731 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.022702 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-service-ca\") pod \"3d191cb0-4c25-4468-b463-b9b816bfd199\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " Sep 10 14:44:20.022943 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.022748 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-oauth-serving-cert\") pod \"3d191cb0-4c25-4468-b463-b9b816bfd199\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " Sep 10 14:44:20.022943 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.022791 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-console-config\") pod \"3d191cb0-4c25-4468-b463-b9b816bfd199\" (UID: \"3d191cb0-4c25-4468-b463-b9b816bfd199\") " Sep 10 14:44:20.023089 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.023057 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3d191cb0-4c25-4468-b463-b9b816bfd199" (UID: "3d191cb0-4c25-4468-b463-b9b816bfd199"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:20.023168 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.023135 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-service-ca" (OuterVolumeSpecName: "service-ca") pod "3d191cb0-4c25-4468-b463-b9b816bfd199" (UID: "3d191cb0-4c25-4468-b463-b9b816bfd199"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:20.023240 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.023150 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3d191cb0-4c25-4468-b463-b9b816bfd199" (UID: "3d191cb0-4c25-4468-b463-b9b816bfd199"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:20.023308 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.023273 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-console-config" (OuterVolumeSpecName: "console-config") pod "3d191cb0-4c25-4468-b463-b9b816bfd199" (UID: "3d191cb0-4c25-4468-b463-b9b816bfd199"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:20.025084 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.025051 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3d191cb0-4c25-4468-b463-b9b816bfd199" (UID: "3d191cb0-4c25-4468-b463-b9b816bfd199"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:20.025084 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.025069 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3d191cb0-4c25-4468-b463-b9b816bfd199" (UID: "3d191cb0-4c25-4468-b463-b9b816bfd199"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:20.025084 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.025055 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d191cb0-4c25-4468-b463-b9b816bfd199-kube-api-access-v5fmx" (OuterVolumeSpecName: "kube-api-access-v5fmx") pod "3d191cb0-4c25-4468-b463-b9b816bfd199" (UID: "3d191cb0-4c25-4468-b463-b9b816bfd199"). InnerVolumeSpecName "kube-api-access-v5fmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:44:20.123837 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.123784 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-oauth-config\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:20.123837 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.123828 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5fmx\" (UniqueName: \"kubernetes.io/projected/3d191cb0-4c25-4468-b463-b9b816bfd199-kube-api-access-v5fmx\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:20.123837 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.123839 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-trusted-ca-bundle\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:20.123837 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.123849 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d191cb0-4c25-4468-b463-b9b816bfd199-console-serving-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:20.124137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.123861 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:20.124137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.123870 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-oauth-serving-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:20.124137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.123878 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d191cb0-4c25-4468-b463-b9b816bfd199-console-config\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:20.533085 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.532762 2584 generic.go:356] "Generic (PLEG): container finished" podID="3d191cb0-4c25-4468-b463-b9b816bfd199" containerID="ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a" exitCode=0 Sep 10 14:44:20.533085 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.532885 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b945f867d-lvrvx" event={"ID":"3d191cb0-4c25-4468-b463-b9b816bfd199","Type":"ContainerDied","Data":"ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a"} Sep 10 14:44:20.533085 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.532939 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b945f867d-lvrvx" event={"ID":"3d191cb0-4c25-4468-b463-b9b816bfd199","Type":"ContainerDied","Data":"f8c278ee78dff5c2dd6ef4f18fcdfaf7be1b7a58c538580198e91e2c056e35f6"} Sep 10 14:44:20.533085 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.532968 2584 scope.go:117] "RemoveContainer" containerID="ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a" Sep 10 14:44:20.533473 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.533148 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b945f867d-lvrvx" Sep 10 14:44:20.542300 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.542266 2584 scope.go:117] "RemoveContainer" containerID="ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a" Sep 10 14:44:20.542653 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:44:20.542624 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a\": container with ID starting with ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a not found: ID does not exist" containerID="ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a" Sep 10 14:44:20.542749 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.542658 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a"} err="failed to get container status \"ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a\": rpc error: code = NotFound desc = could not find container \"ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a\": container with ID starting with ebc1524a17d8c71c691d73c48b08dc25a8c8bab709917eb7cc9f02bbbe95382a not found: ID does not exist" Sep 10 14:44:20.556465 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.556432 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b945f867d-lvrvx"] Sep 10 14:44:20.559930 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:20.559884 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b945f867d-lvrvx"] Sep 10 14:44:21.764696 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:21.764311 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d191cb0-4c25-4468-b463-b9b816bfd199" path="/var/lib/kubelet/pods/3d191cb0-4c25-4468-b463-b9b816bfd199/volumes" Sep 10 14:44:39.423582 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:39.423543 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:39.443430 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:39.443405 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:39.617920 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:39.617892 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:57.476848 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.476818 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:44:57.477380 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.477096 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="prometheus" containerID="cri-o://e198fe4b16bbfa1f6e62307782d6211594a80bbbc8f28f0e8c62bd45ea6f135d" gracePeriod=600 Sep 10 14:44:57.477380 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.477116 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy" containerID="cri-o://4aac8073f77602622390185b7d687cf81cb9925a17672256397d1a51691a119a" gracePeriod=600 Sep 10 14:44:57.477380 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.477167 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-web" containerID="cri-o://79c79e16c0183ec5001542943ad6446b4558400ef791fce2f695fe99777cf780" gracePeriod=600 Sep 10 14:44:57.477380 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.477176 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1cf8c73a224a34f0621b8b681becd2943c1bf5565790ec11c059fc4f05fb8b99" gracePeriod=600 Sep 10 14:44:57.477380 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.477134 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="config-reloader" containerID="cri-o://41fba61e646e0a1e2cb8b4704e8df9cc98b42561fb59fa11502bec3a08434deb" gracePeriod=600 Sep 10 14:44:57.477380 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.477133 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="thanos-sidecar" containerID="cri-o://9c2363b4beaed0e785d3945144f1902f181eb0edea6e66dfaacb0f67cc92d807" gracePeriod=600 Sep 10 14:44:57.668190 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668162 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b631b22-06ef-421f-87cf-a395b0849887" containerID="1cf8c73a224a34f0621b8b681becd2943c1bf5565790ec11c059fc4f05fb8b99" exitCode=0 Sep 10 14:44:57.668190 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668185 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b631b22-06ef-421f-87cf-a395b0849887" containerID="4aac8073f77602622390185b7d687cf81cb9925a17672256397d1a51691a119a" exitCode=0 Sep 10 14:44:57.668190 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668191 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b631b22-06ef-421f-87cf-a395b0849887" containerID="79c79e16c0183ec5001542943ad6446b4558400ef791fce2f695fe99777cf780" exitCode=0 Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668214 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b631b22-06ef-421f-87cf-a395b0849887" containerID="9c2363b4beaed0e785d3945144f1902f181eb0edea6e66dfaacb0f67cc92d807" exitCode=0 Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668223 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b631b22-06ef-421f-87cf-a395b0849887" containerID="41fba61e646e0a1e2cb8b4704e8df9cc98b42561fb59fa11502bec3a08434deb" exitCode=0 Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668231 2584 generic.go:356] "Generic (PLEG): container finished" podID="6b631b22-06ef-421f-87cf-a395b0849887" containerID="e198fe4b16bbfa1f6e62307782d6211594a80bbbc8f28f0e8c62bd45ea6f135d" exitCode=0 Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668240 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"1cf8c73a224a34f0621b8b681becd2943c1bf5565790ec11c059fc4f05fb8b99"} Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668311 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"4aac8073f77602622390185b7d687cf81cb9925a17672256397d1a51691a119a"} Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668337 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"79c79e16c0183ec5001542943ad6446b4558400ef791fce2f695fe99777cf780"} Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668361 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"9c2363b4beaed0e785d3945144f1902f181eb0edea6e66dfaacb0f67cc92d807"} Sep 10 14:44:57.668386 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668385 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"41fba61e646e0a1e2cb8b4704e8df9cc98b42561fb59fa11502bec3a08434deb"} Sep 10 14:44:57.668618 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.668408 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"e198fe4b16bbfa1f6e62307782d6211594a80bbbc8f28f0e8c62bd45ea6f135d"} Sep 10 14:44:57.715611 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.715590 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:57.891311 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891241 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891311 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891299 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-serving-certs-ca-bundle\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891317 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-kube-rbac-proxy\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891334 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-grpc-tls\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891463 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-trusted-ca-bundle\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891497 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-2\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891527 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-web-config\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891556 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-tls-assets\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891595 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-config\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891625 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7dzd\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-kube-api-access-k7dzd\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.891791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891638 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:57.891791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891647 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-db\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891819 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891856 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-tls\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891860 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891879 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891890 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-config-out\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891962 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-0\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.891999 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-thanos-prometheus-http-client-file\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892048 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892027 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-metrics-client-certs\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892481 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892051 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-kubelet-serving-ca-bundle\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892481 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892078 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-metrics-client-ca\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892481 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892112 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-1\") pod \"6b631b22-06ef-421f-87cf-a395b0849887\" (UID: \"6b631b22-06ef-421f-87cf-a395b0849887\") " Sep 10 14:44:57.892481 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892353 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.892481 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892373 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.892481 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892389 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.892779 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.892760 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:57.893402 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.893369 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:44:57.893773 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.893743 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:57.894283 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.894174 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.894283 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.894233 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:57.894283 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.894252 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.894283 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.894257 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:44:57.896292 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896262 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.896384 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896344 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.896384 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896363 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-config" (OuterVolumeSpecName: "config") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.896384 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896362 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-kube-api-access-k7dzd" (OuterVolumeSpecName: "kube-api-access-k7dzd") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "kube-api-access-k7dzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:44:57.896542 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896460 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:44:57.896542 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896460 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-config-out" (OuterVolumeSpecName: "config-out") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:44:57.896542 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896465 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.896850 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.896830 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.898025 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.898009 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.905243 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.905219 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-web-config" (OuterVolumeSpecName: "web-config") pod "6b631b22-06ef-421f-87cf-a395b0849887" (UID: "6b631b22-06ef-421f-87cf-a395b0849887"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 14:44:57.993638 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993619 2584 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-config\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993638 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993637 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7dzd\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-kube-api-access-k7dzd\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993648 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-db\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993657 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993666 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993673 2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6b631b22-06ef-421f-87cf-a395b0849887-config-out\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993681 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993690 2584 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993699 2584 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-metrics-client-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993707 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993716 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-configmap-metrics-client-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993724 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6b631b22-06ef-421f-87cf-a395b0849887-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.993735 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993732 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.994037 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993740 2584 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-kube-rbac-proxy\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.994037 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993748 2584 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-secret-grpc-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.994037 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993756 2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6b631b22-06ef-421f-87cf-a395b0849887-web-config\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:57.994037 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:57.993763 2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6b631b22-06ef-421f-87cf-a395b0849887-tls-assets\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:44:58.674817 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.674788 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6b631b22-06ef-421f-87cf-a395b0849887","Type":"ContainerDied","Data":"68cef6dcc22fd6c682aae8e10ac29527bd72c7eb19079e27c3e9fca39cc47edc"} Sep 10 14:44:58.675151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.674838 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.675151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.674861 2584 scope.go:117] "RemoveContainer" containerID="1cf8c73a224a34f0621b8b681becd2943c1bf5565790ec11c059fc4f05fb8b99" Sep 10 14:44:58.682714 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.682667 2584 scope.go:117] "RemoveContainer" containerID="4aac8073f77602622390185b7d687cf81cb9925a17672256397d1a51691a119a" Sep 10 14:44:58.689364 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.689344 2584 scope.go:117] "RemoveContainer" containerID="79c79e16c0183ec5001542943ad6446b4558400ef791fce2f695fe99777cf780" Sep 10 14:44:58.695459 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.695441 2584 scope.go:117] "RemoveContainer" containerID="9c2363b4beaed0e785d3945144f1902f181eb0edea6e66dfaacb0f67cc92d807" Sep 10 14:44:58.698839 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.698791 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:44:58.702103 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.701951 2584 scope.go:117] "RemoveContainer" containerID="41fba61e646e0a1e2cb8b4704e8df9cc98b42561fb59fa11502bec3a08434deb" Sep 10 14:44:58.704178 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.704160 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:44:58.708273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.708258 2584 scope.go:117] "RemoveContainer" containerID="e198fe4b16bbfa1f6e62307782d6211594a80bbbc8f28f0e8c62bd45ea6f135d" Sep 10 14:44:58.714638 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.714622 2584 scope.go:117] "RemoveContainer" containerID="c4fb097f941352971e3039b906a049075f4f534cc5f47e154f25a9997cc8c357" Sep 10 14:44:58.728114 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728095 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:44:58.728355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728342 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="config-reloader" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728357 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="config-reloader" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728364 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="thanos-sidecar" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728369 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="thanos-sidecar" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728376 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-web" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728382 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-web" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728388 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728393 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728406 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d191cb0-4c25-4468-b463-b9b816bfd199" containerName="console" Sep 10 14:44:58.728411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728411 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d191cb0-4c25-4468-b463-b9b816bfd199" containerName="console" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728418 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-thanos" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728423 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-thanos" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728431 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="init-config-reloader" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728435 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="init-config-reloader" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728443 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="prometheus" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728448 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="prometheus" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728487 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728496 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-web" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728502 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="thanos-sidecar" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728508 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="prometheus" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728513 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="kube-rbac-proxy-thanos" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728519 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d191cb0-4c25-4468-b463-b9b816bfd199" containerName="console" Sep 10 14:44:58.728689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.728524 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b631b22-06ef-421f-87cf-a395b0849887" containerName="config-reloader" Sep 10 14:44:58.731355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.731340 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.734269 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734248 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 10 14:44:58.734374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734294 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 10 14:44:58.734374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734258 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 10 14:44:58.734374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734352 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 10 14:44:58.734530 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734412 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 10 14:44:58.734578 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734540 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 10 14:44:58.734631 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734572 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 10 14:44:58.734815 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734797 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 10 14:44:58.734815 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734807 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1f8cfsfndgdv3\"" Sep 10 14:44:58.734907 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734838 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xj28m\"" Sep 10 14:44:58.734968 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734939 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 10 14:44:58.734968 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734955 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 10 14:44:58.735067 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734996 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 10 14:44:58.735067 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.735005 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 10 14:44:58.735067 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.734942 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 10 14:44:58.737902 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.737881 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 10 14:44:58.740094 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.740053 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 10 14:44:58.748553 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.748535 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:44:58.898012 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.897982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898016 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898044 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898088 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpwx6\" (UniqueName: \"kubernetes.io/projected/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-kube-api-access-wpwx6\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898116 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898159 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898186 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898252 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898277 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898299 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898328 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898348 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898370 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898363 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898623 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898389 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898623 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898407 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898623 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898423 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898623 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898437 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898623 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898452 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.898623 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.898473 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-config\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.998971 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.998918 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.998971 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.998945 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.998971 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.998966 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.998982 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999100 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-config\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999151 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999143 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999168 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999254 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999278 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpwx6\" (UniqueName: \"kubernetes.io/projected/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-kube-api-access-wpwx6\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999310 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999350 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999414 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999440 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999475 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999506 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999538 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999585 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999564 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999961 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999593 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999961 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999773 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:58.999961 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:58.999773 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.000115 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.000018 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.001992 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.001964 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-config-out\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.002077 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.002004 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.002077 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.002058 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.002597 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.002466 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.002597 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.002503 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-web-config\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.003784 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.002818 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.003784 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.002985 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.003784 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.003019 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.003784 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.003249 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.003784 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.003729 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.003784 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.003755 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.004108 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.003877 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.004506 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.004480 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-config\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.004760 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.004726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.004808 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.004726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.005688 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.005671 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.011216 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.011177 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpwx6\" (UniqueName: \"kubernetes.io/projected/3d147cbe-24f9-4db8-86aa-c3ebd7b9817b-kube-api-access-wpwx6\") pod \"prometheus-k8s-0\" (UID: \"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.041076 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.041048 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:44:59.320508 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.320481 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 10 14:44:59.321209 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:44:59.321179 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d147cbe_24f9_4db8_86aa_c3ebd7b9817b.slice/crio-feacc878c3e7e631fb558b43ea1ce2cfe3ff62855304de630e48a3924a5218ad WatchSource:0}: Error finding container feacc878c3e7e631fb558b43ea1ce2cfe3ff62855304de630e48a3924a5218ad: Status 404 returned error can't find the container with id feacc878c3e7e631fb558b43ea1ce2cfe3ff62855304de630e48a3924a5218ad Sep 10 14:44:59.679488 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.679401 2584 generic.go:356] "Generic (PLEG): container finished" podID="3d147cbe-24f9-4db8-86aa-c3ebd7b9817b" containerID="866b9b56d17f56eec47a91dc3b3d27510e8dc7d19231b0b05d6da76ee7d10fee" exitCode=0 Sep 10 14:44:59.679913 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.679490 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerDied","Data":"866b9b56d17f56eec47a91dc3b3d27510e8dc7d19231b0b05d6da76ee7d10fee"} Sep 10 14:44:59.679913 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.679570 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerStarted","Data":"feacc878c3e7e631fb558b43ea1ce2cfe3ff62855304de630e48a3924a5218ad"} Sep 10 14:44:59.761964 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:44:59.761941 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b631b22-06ef-421f-87cf-a395b0849887" path="/var/lib/kubelet/pods/6b631b22-06ef-421f-87cf-a395b0849887/volumes" Sep 10 14:45:00.687821 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:00.687785 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerStarted","Data":"93976c87b706758865f6dc1e5e83d263b5007076b83f6d91efa26db515d53e45"} Sep 10 14:45:00.688302 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:00.687862 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerStarted","Data":"08a38bfb6c39bdbd7015048e9a5d9c87d4cba84b903224704d5a084bb609b66f"} Sep 10 14:45:00.688302 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:00.687913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerStarted","Data":"ff423ea833e8cec9f62b2a2e75479c473bf34dfd01d088599a2dca92445c4b92"} Sep 10 14:45:00.688302 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:00.687958 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerStarted","Data":"e469805b8014428363551b4d8dc8d4e3ef5ea4737a7a04d5cf03f24ed4bbc3cf"} Sep 10 14:45:00.688302 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:00.687998 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerStarted","Data":"56bf19715572170d85e65c2a1517660aa528dacee06de0a37cbb2bfeeaa0e633"} Sep 10 14:45:00.688302 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:00.688035 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3d147cbe-24f9-4db8-86aa-c3ebd7b9817b","Type":"ContainerStarted","Data":"60652ca4cec08065fa07224d93f94f9954e85eaef5087f9c3402edaee671d19c"} Sep 10 14:45:00.718433 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:00.718378 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.698984995 podStartE2EDuration="2.718358683s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.373688ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:45:00.714134716 +0000 UTC m=+157.386160510" watchObservedRunningTime="2026-09-10 14:45:00.718358683 +0000 UTC m=+157.390384455" Sep 10 14:45:04.041169 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:04.041136 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:45:59.041406 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:59.041371 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:45:59.056475 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:59.056444 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:45:59.904266 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:45:59.904241 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 10 14:46:45.737824 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.737790 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n"] Sep 10 14:46:45.740009 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.739993 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:45.742892 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.742868 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 10 14:46:45.743070 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.743052 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2862c\"" Sep 10 14:46:45.744343 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.744325 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 10 14:46:45.752970 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.752940 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n"] Sep 10 14:46:45.771750 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.771724 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/354f19fe-8c46-4c56-9b18-f6ac68b07588-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7r98n\" (UID: \"354f19fe-8c46-4c56-9b18-f6ac68b07588\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:45.771847 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.771770 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqf6t\" (UniqueName: \"kubernetes.io/projected/354f19fe-8c46-4c56-9b18-f6ac68b07588-kube-api-access-kqf6t\") pod \"openshift-lws-operator-fd8ccff4c-7r98n\" (UID: \"354f19fe-8c46-4c56-9b18-f6ac68b07588\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:45.872080 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.872057 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqf6t\" (UniqueName: \"kubernetes.io/projected/354f19fe-8c46-4c56-9b18-f6ac68b07588-kube-api-access-kqf6t\") pod \"openshift-lws-operator-fd8ccff4c-7r98n\" (UID: \"354f19fe-8c46-4c56-9b18-f6ac68b07588\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:45.872160 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.872108 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/354f19fe-8c46-4c56-9b18-f6ac68b07588-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7r98n\" (UID: \"354f19fe-8c46-4c56-9b18-f6ac68b07588\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:45.872409 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.872394 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/354f19fe-8c46-4c56-9b18-f6ac68b07588-tmp\") pod \"openshift-lws-operator-fd8ccff4c-7r98n\" (UID: \"354f19fe-8c46-4c56-9b18-f6ac68b07588\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:45.881948 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:45.881928 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqf6t\" (UniqueName: \"kubernetes.io/projected/354f19fe-8c46-4c56-9b18-f6ac68b07588-kube-api-access-kqf6t\") pod \"openshift-lws-operator-fd8ccff4c-7r98n\" (UID: \"354f19fe-8c46-4c56-9b18-f6ac68b07588\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:46.059137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:46.059066 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" Sep 10 14:46:46.329138 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:46.329041 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n"] Sep 10 14:46:46.332848 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:46:46.332821 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod354f19fe_8c46_4c56_9b18_f6ac68b07588.slice/crio-41143d3ebd1ccd5dc854b53c9ac4fb3fb27d512f4391316b98e89a847c2caf08 WatchSource:0}: Error finding container 41143d3ebd1ccd5dc854b53c9ac4fb3fb27d512f4391316b98e89a847c2caf08: Status 404 returned error can't find the container with id 41143d3ebd1ccd5dc854b53c9ac4fb3fb27d512f4391316b98e89a847c2caf08 Sep 10 14:46:47.054769 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:47.054736 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" event={"ID":"354f19fe-8c46-4c56-9b18-f6ac68b07588","Type":"ContainerStarted","Data":"41143d3ebd1ccd5dc854b53c9ac4fb3fb27d512f4391316b98e89a847c2caf08"} Sep 10 14:46:50.067297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:50.067261 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" event={"ID":"354f19fe-8c46-4c56-9b18-f6ac68b07588","Type":"ContainerStarted","Data":"65b277194673c2594a24480074d3dcd09114832c98620f43936bdda102e6cb8b"} Sep 10 14:46:50.082962 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:46:50.082809 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-7r98n" podStartSLOduration=2.345777568 podStartE2EDuration="5.082789494s" totalImagesPullingTime="2.737011926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:46:50.082340191 +0000 UTC m=+266.754365963" watchObservedRunningTime="2026-09-10 14:46:50.082789494 +0000 UTC m=+266.754815268" Sep 10 14:47:05.226002 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.225964 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd"] Sep 10 14:47:05.234342 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.234311 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.238137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.238107 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 10 14:47:05.238310 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.238107 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 10 14:47:05.238418 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.238402 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7smlt\"" Sep 10 14:47:05.238486 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.238452 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 10 14:47:05.239102 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.239084 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 10 14:47:05.248043 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.248016 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd"] Sep 10 14:47:05.318071 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.318023 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dwtl\" (UniqueName: \"kubernetes.io/projected/d0e87f50-d14f-4e0c-8b7d-94b16320033a-kube-api-access-6dwtl\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.318292 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.318081 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0e87f50-d14f-4e0c-8b7d-94b16320033a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.318292 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.318167 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0e87f50-d14f-4e0c-8b7d-94b16320033a-tmp\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.318292 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.318253 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0e87f50-d14f-4e0c-8b7d-94b16320033a-manifests\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.318292 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.318288 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e87f50-d14f-4e0c-8b7d-94b16320033a-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.419451 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.419408 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dwtl\" (UniqueName: \"kubernetes.io/projected/d0e87f50-d14f-4e0c-8b7d-94b16320033a-kube-api-access-6dwtl\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.419645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.419460 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0e87f50-d14f-4e0c-8b7d-94b16320033a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.419645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.419574 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0e87f50-d14f-4e0c-8b7d-94b16320033a-tmp\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.419645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.419627 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0e87f50-d14f-4e0c-8b7d-94b16320033a-manifests\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.419829 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.419660 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e87f50-d14f-4e0c-8b7d-94b16320033a-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.420089 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.420057 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d0e87f50-d14f-4e0c-8b7d-94b16320033a-manifests\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.421966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.421939 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0e87f50-d14f-4e0c-8b7d-94b16320033a-tmp\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.422153 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.422130 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d0e87f50-d14f-4e0c-8b7d-94b16320033a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.422215 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.422137 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0e87f50-d14f-4e0c-8b7d-94b16320033a-webhook-cert\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.427545 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.427515 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dwtl\" (UniqueName: \"kubernetes.io/projected/d0e87f50-d14f-4e0c-8b7d-94b16320033a-kube-api-access-6dwtl\") pod \"opendatahub-operator-controller-manager-68f84f89d7-258zd\" (UID: \"d0e87f50-d14f-4e0c-8b7d-94b16320033a\") " pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.546390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.546299 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:05.830074 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:05.830032 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd"] Sep 10 14:47:05.832717 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:47:05.832690 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e87f50_d14f_4e0c_8b7d_94b16320033a.slice/crio-e8ae2ee61e4bd28c30e16c3881a413705ebdb4ca63ce5822cf32d74f65810fc7 WatchSource:0}: Error finding container e8ae2ee61e4bd28c30e16c3881a413705ebdb4ca63ce5822cf32d74f65810fc7: Status 404 returned error can't find the container with id e8ae2ee61e4bd28c30e16c3881a413705ebdb4ca63ce5822cf32d74f65810fc7 Sep 10 14:47:06.125311 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:06.125222 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" event={"ID":"d0e87f50-d14f-4e0c-8b7d-94b16320033a","Type":"ContainerStarted","Data":"e8ae2ee61e4bd28c30e16c3881a413705ebdb4ca63ce5822cf32d74f65810fc7"} Sep 10 14:47:09.141182 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:09.141144 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" event={"ID":"d0e87f50-d14f-4e0c-8b7d-94b16320033a","Type":"ContainerStarted","Data":"94a447df4bfd9f5878638711846b4c9cca53ed7fda7b4283b35d12bd435789ca"} Sep 10 14:47:11.150430 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:11.150394 2584 generic.go:356] "Generic (PLEG): container finished" podID="d0e87f50-d14f-4e0c-8b7d-94b16320033a" containerID="94a447df4bfd9f5878638711846b4c9cca53ed7fda7b4283b35d12bd435789ca" exitCode=0 Sep 10 14:47:11.150812 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:11.150461 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" event={"ID":"d0e87f50-d14f-4e0c-8b7d-94b16320033a","Type":"ContainerDied","Data":"94a447df4bfd9f5878638711846b4c9cca53ed7fda7b4283b35d12bd435789ca"} Sep 10 14:47:12.156094 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:12.156055 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" event={"ID":"d0e87f50-d14f-4e0c-8b7d-94b16320033a","Type":"ContainerStarted","Data":"e9bdcadcec840a8ee03f861d2883d3ed962268dca3ea51d4bc957446c057e2f3"} Sep 10 14:47:12.156450 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:12.156283 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:12.179054 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:12.179009 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" podStartSLOduration=2.527561313 podStartE2EDuration="7.178997041s" totalImagesPullingTime="3.57782042s" totalInitContainerRuntime="1.073615308s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:12.177301574 +0000 UTC m=+288.849327344" watchObservedRunningTime="2026-09-10 14:47:12.178997041 +0000 UTC m=+288.851022811" Sep 10 14:47:23.164193 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.164155 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-68f84f89d7-258zd" Sep 10 14:47:23.442354 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.442327 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv"] Sep 10 14:47:23.518100 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.518064 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv"] Sep 10 14:47:23.518306 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.518209 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.521075 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.521026 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8jf4z\"" Sep 10 14:47:23.521075 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.521058 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 14:47:23.521319 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.521110 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 14:47:23.521319 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.521239 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 14:47:23.558712 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.558674 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-cert\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.558882 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.558727 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-manager-config\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.558882 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.558772 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-metrics-cert\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.558882 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.558809 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqgr4\" (UniqueName: \"kubernetes.io/projected/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-kube-api-access-nqgr4\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.659872 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.659843 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-cert\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.660055 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.659881 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-manager-config\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.660055 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.659915 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-metrics-cert\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.660055 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.659976 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqgr4\" (UniqueName: \"kubernetes.io/projected/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-kube-api-access-nqgr4\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.663667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.663636 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 10 14:47:23.664045 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.664017 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 10 14:47:23.664434 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.664413 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 10 14:47:23.671221 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.670061 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqgr4\" (UniqueName: \"kubernetes.io/projected/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-kube-api-access-nqgr4\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.671221 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.670784 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-manager-config\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.672946 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.672918 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-metrics-cert\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.673117 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.673093 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c1351a7-f82b-4bbd-bf6a-31000e0c8843-cert\") pod \"lws-controller-manager-bc768b6-8wjxv\" (UID: \"1c1351a7-f82b-4bbd-bf6a-31000e0c8843\") " pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:23.689728 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.689697 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:47:23.690268 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.690249 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:47:23.696974 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.696855 2584 kubelet.go:1648] "Image garbage collection succeeded" Sep 10 14:47:23.830683 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.830513 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8jf4z\"" Sep 10 14:47:23.987017 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:23.838080 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:24.126771 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:24.126738 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv"] Sep 10 14:47:24.130784 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:47:24.130754 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c1351a7_f82b_4bbd_bf6a_31000e0c8843.slice/crio-bc61df20e75683e2116aff33e1c9c2f037c283414e1102b6718b295de7e9735d WatchSource:0}: Error finding container bc61df20e75683e2116aff33e1c9c2f037c283414e1102b6718b295de7e9735d: Status 404 returned error can't find the container with id bc61df20e75683e2116aff33e1c9c2f037c283414e1102b6718b295de7e9735d Sep 10 14:47:24.205676 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:24.205638 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" event={"ID":"1c1351a7-f82b-4bbd-bf6a-31000e0c8843","Type":"ContainerStarted","Data":"bc61df20e75683e2116aff33e1c9c2f037c283414e1102b6718b295de7e9735d"} Sep 10 14:47:26.220578 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:26.220543 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" event={"ID":"1c1351a7-f82b-4bbd-bf6a-31000e0c8843","Type":"ContainerStarted","Data":"9e2c7b08ada8a973922cf7a38c8866e16d832cbe2a05c40b332f930a1ac030ce"} Sep 10 14:47:26.220951 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:26.220690 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:26.242745 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:26.242700 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" podStartSLOduration=1.614769537 podStartE2EDuration="3.242688076s" totalImagesPullingTime="1.627918539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:26.241731831 +0000 UTC m=+302.913757614" watchObservedRunningTime="2026-09-10 14:47:26.242688076 +0000 UTC m=+302.914713849" Sep 10 14:47:33.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.687541 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc"] Sep 10 14:47:33.690240 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.690223 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.693480 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.693457 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 10 14:47:33.693648 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.693606 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zrv8g\"" Sep 10 14:47:33.693714 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.693686 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 10 14:47:33.694808 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.694790 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 10 14:47:33.714932 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.714905 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc"] Sep 10 14:47:33.840374 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840340 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/552a922a-babc-46d8-8e08-e59ee9704a80-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840536 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840400 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840536 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840467 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/552a922a-babc-46d8-8e08-e59ee9704a80-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840536 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840492 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqbsf\" (UniqueName: \"kubernetes.io/projected/552a922a-babc-46d8-8e08-e59ee9704a80-kube-api-access-vqbsf\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840637 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840564 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840637 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840609 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840643 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840667 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.840701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.840692 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/552a922a-babc-46d8-8e08-e59ee9704a80-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941318 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941289 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941496 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941328 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/552a922a-babc-46d8-8e08-e59ee9704a80-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941496 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941344 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqbsf\" (UniqueName: \"kubernetes.io/projected/552a922a-babc-46d8-8e08-e59ee9704a80-kube-api-access-vqbsf\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941496 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941366 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941496 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941424 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941496 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941474 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941754 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941497 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941754 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941522 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/552a922a-babc-46d8-8e08-e59ee9704a80-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941754 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941564 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/552a922a-babc-46d8-8e08-e59ee9704a80-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941754 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941724 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.941927 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.942009 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941919 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.942009 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.941967 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.942385 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.942366 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/552a922a-babc-46d8-8e08-e59ee9704a80-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.943892 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.943865 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/552a922a-babc-46d8-8e08-e59ee9704a80-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.943991 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.943950 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/552a922a-babc-46d8-8e08-e59ee9704a80-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.949889 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.949831 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqbsf\" (UniqueName: \"kubernetes.io/projected/552a922a-babc-46d8-8e08-e59ee9704a80-kube-api-access-vqbsf\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:33.950003 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:33.949985 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/552a922a-babc-46d8-8e08-e59ee9704a80-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc\" (UID: \"552a922a-babc-46d8-8e08-e59ee9704a80\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:34.000070 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:34.000037 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:34.286817 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:34.286780 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc"] Sep 10 14:47:34.289269 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:47:34.289238 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552a922a_babc_46d8_8e08_e59ee9704a80.slice/crio-41c5d1a715dbf917aac03bf4765bf0e6073ef4b1e609f769bbee24cdf880b1d0 WatchSource:0}: Error finding container 41c5d1a715dbf917aac03bf4765bf0e6073ef4b1e609f769bbee24cdf880b1d0: Status 404 returned error can't find the container with id 41c5d1a715dbf917aac03bf4765bf0e6073ef4b1e609f769bbee24cdf880b1d0 Sep 10 14:47:34.291269 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:34.291252 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 14:47:35.258649 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:35.258617 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" event={"ID":"552a922a-babc-46d8-8e08-e59ee9704a80","Type":"ContainerStarted","Data":"41c5d1a715dbf917aac03bf4765bf0e6073ef4b1e609f769bbee24cdf880b1d0"} Sep 10 14:47:36.193950 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.193910 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dhxjp"] Sep 10 14:47:36.201134 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.201107 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.204733 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.204705 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 10 14:47:36.204847 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.204809 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9vn6g\"" Sep 10 14:47:36.211061 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.211035 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dhxjp"] Sep 10 14:47:36.260791 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.260755 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2da0e7e-ad04-4157-ac82-8bbb34026a15-cert\") pod \"odh-model-controller-7998c67dc4-dhxjp\" (UID: \"c2da0e7e-ad04-4157-ac82-8bbb34026a15\") " pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.261356 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.260859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-668ts\" (UniqueName: \"kubernetes.io/projected/c2da0e7e-ad04-4157-ac82-8bbb34026a15-kube-api-access-668ts\") pod \"odh-model-controller-7998c67dc4-dhxjp\" (UID: \"c2da0e7e-ad04-4157-ac82-8bbb34026a15\") " pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.362383 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.362336 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668ts\" (UniqueName: \"kubernetes.io/projected/c2da0e7e-ad04-4157-ac82-8bbb34026a15-kube-api-access-668ts\") pod \"odh-model-controller-7998c67dc4-dhxjp\" (UID: \"c2da0e7e-ad04-4157-ac82-8bbb34026a15\") " pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.362559 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.362413 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2da0e7e-ad04-4157-ac82-8bbb34026a15-cert\") pod \"odh-model-controller-7998c67dc4-dhxjp\" (UID: \"c2da0e7e-ad04-4157-ac82-8bbb34026a15\") " pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.362559 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:47:36.362543 2584 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 10 14:47:36.362685 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:47:36.362610 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2da0e7e-ad04-4157-ac82-8bbb34026a15-cert podName:c2da0e7e-ad04-4157-ac82-8bbb34026a15 nodeName:}" failed. No retries permitted until 2026-09-10 14:47:36.862588389 +0000 UTC m=+313.534614142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c2da0e7e-ad04-4157-ac82-8bbb34026a15-cert") pod "odh-model-controller-7998c67dc4-dhxjp" (UID: "c2da0e7e-ad04-4157-ac82-8bbb34026a15") : secret "odh-model-controller-webhook-cert" not found Sep 10 14:47:36.373289 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.373257 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-668ts\" (UniqueName: \"kubernetes.io/projected/c2da0e7e-ad04-4157-ac82-8bbb34026a15-kube-api-access-668ts\") pod \"odh-model-controller-7998c67dc4-dhxjp\" (UID: \"c2da0e7e-ad04-4157-ac82-8bbb34026a15\") " pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.847268 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.847232 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:47:36.847355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.847304 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:47:36.847355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.847335 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:47:36.866528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.866504 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2da0e7e-ad04-4157-ac82-8bbb34026a15-cert\") pod \"odh-model-controller-7998c67dc4-dhxjp\" (UID: \"c2da0e7e-ad04-4157-ac82-8bbb34026a15\") " pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.868616 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.868597 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c2da0e7e-ad04-4157-ac82-8bbb34026a15-cert\") pod \"odh-model-controller-7998c67dc4-dhxjp\" (UID: \"c2da0e7e-ad04-4157-ac82-8bbb34026a15\") " pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:36.931411 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.931386 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wk6mv"] Sep 10 14:47:36.935031 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.935012 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:36.938273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.938251 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2pchh\"" Sep 10 14:47:36.938472 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.938251 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 10 14:47:36.945543 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:36.945518 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wk6mv"] Sep 10 14:47:37.068707 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.068674 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06adc0df-7271-4324-ba26-94e3847000e9-tls-certs\") pod \"model-serving-api-54d89bd5fc-wk6mv\" (UID: \"06adc0df-7271-4324-ba26-94e3847000e9\") " pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:37.068911 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.068714 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcvzf\" (UniqueName: \"kubernetes.io/projected/06adc0df-7271-4324-ba26-94e3847000e9-kube-api-access-mcvzf\") pod \"model-serving-api-54d89bd5fc-wk6mv\" (UID: \"06adc0df-7271-4324-ba26-94e3847000e9\") " pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:37.113421 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.113334 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:37.170172 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.170081 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06adc0df-7271-4324-ba26-94e3847000e9-tls-certs\") pod \"model-serving-api-54d89bd5fc-wk6mv\" (UID: \"06adc0df-7271-4324-ba26-94e3847000e9\") " pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:37.170172 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.170145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcvzf\" (UniqueName: \"kubernetes.io/projected/06adc0df-7271-4324-ba26-94e3847000e9-kube-api-access-mcvzf\") pod \"model-serving-api-54d89bd5fc-wk6mv\" (UID: \"06adc0df-7271-4324-ba26-94e3847000e9\") " pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:37.170670 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:47:37.170640 2584 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Sep 10 14:47:37.170751 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:47:37.170712 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06adc0df-7271-4324-ba26-94e3847000e9-tls-certs podName:06adc0df-7271-4324-ba26-94e3847000e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:47:37.670690897 +0000 UTC m=+314.342716647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/06adc0df-7271-4324-ba26-94e3847000e9-tls-certs") pod "model-serving-api-54d89bd5fc-wk6mv" (UID: "06adc0df-7271-4324-ba26-94e3847000e9") : secret "model-serving-api-tls" not found Sep 10 14:47:37.182422 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.182396 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcvzf\" (UniqueName: \"kubernetes.io/projected/06adc0df-7271-4324-ba26-94e3847000e9-kube-api-access-mcvzf\") pod \"model-serving-api-54d89bd5fc-wk6mv\" (UID: \"06adc0df-7271-4324-ba26-94e3847000e9\") " pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:37.226884 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.226857 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bc768b6-8wjxv" Sep 10 14:47:37.272528 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.272485 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" event={"ID":"552a922a-babc-46d8-8e08-e59ee9704a80","Type":"ContainerStarted","Data":"294009de8927f5dff6ce5fc957b04d0c6b174970067874699e36b07653cda635"} Sep 10 14:47:37.293690 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.293645 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" podStartSLOduration=1.738053634 podStartE2EDuration="4.293629858s" totalImagesPullingTime="2.555576224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:37.292459144 +0000 UTC m=+313.964484916" watchObservedRunningTime="2026-09-10 14:47:37.293629858 +0000 UTC m=+313.965655630" Sep 10 14:47:37.389838 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.389770 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dhxjp"] Sep 10 14:47:37.393358 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:47:37.393321 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2da0e7e_ad04_4157_ac82_8bbb34026a15.slice/crio-c69b2e9e65a5755e4d1bb524d392c599c815283fe5c6c8193b07dd4ac812bbd0 WatchSource:0}: Error finding container c69b2e9e65a5755e4d1bb524d392c599c815283fe5c6c8193b07dd4ac812bbd0: Status 404 returned error can't find the container with id c69b2e9e65a5755e4d1bb524d392c599c815283fe5c6c8193b07dd4ac812bbd0 Sep 10 14:47:37.675368 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.675277 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06adc0df-7271-4324-ba26-94e3847000e9-tls-certs\") pod \"model-serving-api-54d89bd5fc-wk6mv\" (UID: \"06adc0df-7271-4324-ba26-94e3847000e9\") " pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:37.677706 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.677674 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06adc0df-7271-4324-ba26-94e3847000e9-tls-certs\") pod \"model-serving-api-54d89bd5fc-wk6mv\" (UID: \"06adc0df-7271-4324-ba26-94e3847000e9\") " pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:37.846578 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:37.846546 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:38.000774 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:38.000747 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:38.005056 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:38.005038 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:38.115527 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:38.115422 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-wk6mv"] Sep 10 14:47:38.118073 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:47:38.118037 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06adc0df_7271_4324_ba26_94e3847000e9.slice/crio-a96930bcdf7b9c7e91817a177364c541b3548e1b3aa85b7fb79a6d7a7ef72ced WatchSource:0}: Error finding container a96930bcdf7b9c7e91817a177364c541b3548e1b3aa85b7fb79a6d7a7ef72ced: Status 404 returned error can't find the container with id a96930bcdf7b9c7e91817a177364c541b3548e1b3aa85b7fb79a6d7a7ef72ced Sep 10 14:47:38.279410 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:38.279276 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" event={"ID":"06adc0df-7271-4324-ba26-94e3847000e9","Type":"ContainerStarted","Data":"a96930bcdf7b9c7e91817a177364c541b3548e1b3aa85b7fb79a6d7a7ef72ced"} Sep 10 14:47:38.281700 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:38.281667 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" event={"ID":"c2da0e7e-ad04-4157-ac82-8bbb34026a15","Type":"ContainerStarted","Data":"c69b2e9e65a5755e4d1bb524d392c599c815283fe5c6c8193b07dd4ac812bbd0"} Sep 10 14:47:38.282139 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:38.282116 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:38.283227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:38.283187 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc" Sep 10 14:47:42.302719 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.302681 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" event={"ID":"06adc0df-7271-4324-ba26-94e3847000e9","Type":"ContainerStarted","Data":"1af8ed4000934cb6b1926739bc327471b558e4d0bc60a97dfd70e0c053989be8"} Sep 10 14:47:42.303281 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.302804 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:42.304265 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.304246 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" event={"ID":"c2da0e7e-ad04-4157-ac82-8bbb34026a15","Type":"ContainerStarted","Data":"49c22ada3b699d4518168256907574305174d2ce2a1566ceaa707278e4b3a71e"} Sep 10 14:47:42.304319 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.304297 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:47:42.323737 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.323696 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" podStartSLOduration=2.896741922 podStartE2EDuration="6.323684423s" totalImagesPullingTime="3.426942501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:42.322279913 +0000 UTC m=+318.994305686" watchObservedRunningTime="2026-09-10 14:47:42.323684423 +0000 UTC m=+318.995710194" Sep 10 14:47:42.341207 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.341163 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" podStartSLOduration=2.229519516 podStartE2EDuration="6.341151008s" totalImagesPullingTime="4.111631492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:42.341000507 +0000 UTC m=+319.013026279" watchObservedRunningTime="2026-09-10 14:47:42.341151008 +0000 UTC m=+319.013176780" Sep 10 14:47:42.537102 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.537070 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc"] Sep 10 14:47:42.540409 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.540394 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.543489 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.543451 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 10 14:47:42.543489 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.543468 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 10 14:47:42.543655 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.543511 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 10 14:47:42.543796 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.543777 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vbqdk\"" Sep 10 14:47:42.556237 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.556172 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc"] Sep 10 14:47:42.620014 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.619984 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.620140 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.620016 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.620140 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.620076 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb2mj\" (UniqueName: \"kubernetes.io/projected/5538635f-c690-4e2a-aaf8-1fdfc90b7059-kube-api-access-xb2mj\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.620140 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.620110 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.620340 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.620181 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.620340 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.620236 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.620340 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.620293 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538635f-c690-4e2a-aaf8-1fdfc90b7059-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.720666 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.720641 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.720781 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.720670 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.720781 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.720697 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538635f-c690-4e2a-aaf8-1fdfc90b7059-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.720781 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.720736 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.720781 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.720753 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.720976 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.720790 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb2mj\" (UniqueName: \"kubernetes.io/projected/5538635f-c690-4e2a-aaf8-1fdfc90b7059-kube-api-access-xb2mj\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.720976 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.720818 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.721619 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.721592 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.723107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.723080 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5538635f-c690-4e2a-aaf8-1fdfc90b7059-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.723227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.723122 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.723227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.723123 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.723227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.723215 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.728818 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.728795 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb2mj\" (UniqueName: \"kubernetes.io/projected/5538635f-c690-4e2a-aaf8-1fdfc90b7059-kube-api-access-xb2mj\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.728918 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.728884 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5538635f-c690-4e2a-aaf8-1fdfc90b7059-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-fhwjc\" (UID: \"5538635f-c690-4e2a-aaf8-1fdfc90b7059\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:42.849510 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:42.849449 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:43.138375 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:43.138353 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc"] Sep 10 14:47:43.140672 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:47:43.140643 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5538635f_c690_4e2a_aaf8_1fdfc90b7059.slice/crio-32d7c9438fa4578f9407c5330275123bfd24dca3194c8e61badf698e2f008e35 WatchSource:0}: Error finding container 32d7c9438fa4578f9407c5330275123bfd24dca3194c8e61badf698e2f008e35: Status 404 returned error can't find the container with id 32d7c9438fa4578f9407c5330275123bfd24dca3194c8e61badf698e2f008e35 Sep 10 14:47:43.310415 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:43.310377 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" event={"ID":"5538635f-c690-4e2a-aaf8-1fdfc90b7059","Type":"ContainerStarted","Data":"32d7c9438fa4578f9407c5330275123bfd24dca3194c8e61badf698e2f008e35"} Sep 10 14:47:45.725810 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:45.725771 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:47:45.726119 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:45.725841 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:47:46.328590 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:46.328545 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" event={"ID":"5538635f-c690-4e2a-aaf8-1fdfc90b7059","Type":"ContainerStarted","Data":"1e91761e44624c7860a9e202f6ce508c176902109849dd4984df65a7ba94f65a"} Sep 10 14:47:46.329154 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:46.329128 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:46.330908 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:46.330876 2584 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-fhwjc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 14:47:46.331057 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:46.330937 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" podUID="5538635f-c690-4e2a-aaf8-1fdfc90b7059" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 14:47:46.366317 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:46.366265 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" podStartSLOduration=1.783543629 podStartE2EDuration="4.366248325s" totalImagesPullingTime="2.582704696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:46.365804887 +0000 UTC m=+323.037830661" watchObservedRunningTime="2026-09-10 14:47:46.366248325 +0000 UTC m=+323.038274098" Sep 10 14:47:47.334220 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:47.334184 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-fhwjc" Sep 10 14:47:53.314831 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:53.314805 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-wk6mv" Sep 10 14:47:53.349677 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:47:53.349646 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dhxjp" Sep 10 14:49:55.358248 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.358147 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd"] Sep 10 14:49:55.361314 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.361297 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.364027 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.363996 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-r56xh\"" Sep 10 14:49:55.374694 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.374666 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd"] Sep 10 14:49:55.468456 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468422 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468608 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468469 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468608 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468542 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468608 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468572 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm9c7\" (UniqueName: \"kubernetes.io/projected/a1509437-33ca-470a-8b3b-4078375ad640-kube-api-access-nm9c7\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468608 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468607 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1509437-33ca-470a-8b3b-4078375ad640-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468770 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468687 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1509437-33ca-470a-8b3b-4078375ad640-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468770 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468727 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1509437-33ca-470a-8b3b-4078375ad640-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468770 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.468877 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.468787 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569282 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1509437-33ca-470a-8b3b-4078375ad640-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569423 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569298 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1509437-33ca-470a-8b3b-4078375ad640-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569423 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569317 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1509437-33ca-470a-8b3b-4078375ad640-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569423 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569349 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569423 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569423 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569412 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569655 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569444 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569655 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569479 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569655 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569505 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9c7\" (UniqueName: \"kubernetes.io/projected/a1509437-33ca-470a-8b3b-4078375ad640-kube-api-access-nm9c7\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569870 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569848 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.569924 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.569875 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.570029 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.570005 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.570107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.570050 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.570107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.570060 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1509437-33ca-470a-8b3b-4078375ad640-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.571770 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.571744 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1509437-33ca-470a-8b3b-4078375ad640-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.572073 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.572057 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1509437-33ca-470a-8b3b-4078375ad640-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.577269 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.577247 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm9c7\" (UniqueName: \"kubernetes.io/projected/a1509437-33ca-470a-8b3b-4078375ad640-kube-api-access-nm9c7\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.577367 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.577316 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1509437-33ca-470a-8b3b-4078375ad640-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rbmhd\" (UID: \"a1509437-33ca-470a-8b3b-4078375ad640\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.671004 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.670944 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:55.944229 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.944111 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd"] Sep 10 14:49:55.946663 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:49:55.946631 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1509437_33ca_470a_8b3b_4078375ad640.slice/crio-266960b4b38ab48fb826a0476a206efd750366a0295c15e194d47f906b398654 WatchSource:0}: Error finding container 266960b4b38ab48fb826a0476a206efd750366a0295c15e194d47f906b398654: Status 404 returned error can't find the container with id 266960b4b38ab48fb826a0476a206efd750366a0295c15e194d47f906b398654 Sep 10 14:49:55.948595 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.948560 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:49:55.948688 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.948619 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:49:55.948688 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:55.948647 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:49:56.872994 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:56.872960 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" event={"ID":"a1509437-33ca-470a-8b3b-4078375ad640","Type":"ContainerStarted","Data":"b68e645d89bbabc09a0d4b4f49cfb4dcabe8ba639afd68924512115c27578821"} Sep 10 14:49:56.873416 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:56.873017 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" event={"ID":"a1509437-33ca-470a-8b3b-4078375ad640","Type":"ContainerStarted","Data":"266960b4b38ab48fb826a0476a206efd750366a0295c15e194d47f906b398654"} Sep 10 14:49:56.895470 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:56.895424 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" podStartSLOduration=1.895408508 podStartE2EDuration="1.895408508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:49:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:49:56.892895021 +0000 UTC m=+453.564920793" watchObservedRunningTime="2026-09-10 14:49:56.895408508 +0000 UTC m=+453.567434352" Sep 10 14:49:57.671564 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:57.671528 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:57.676241 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:57.676193 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:57.877489 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:57.877464 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:49:57.878287 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:49:57.878271 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:51:13.400491 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.400461 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c"] Sep 10 14:51:13.403741 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.403725 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.407479 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.407458 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 10 14:51:13.407568 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.407519 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-c8twj\"" Sep 10 14:51:13.417142 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.417101 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c"] Sep 10 14:51:13.435992 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.435965 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3425b6dc-5b64-42b6-991d-d645eef00447-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2rh5c\" (UID: \"3425b6dc-5b64-42b6-991d-d645eef00447\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.436109 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.436007 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2j9z\" (UniqueName: \"kubernetes.io/projected/3425b6dc-5b64-42b6-991d-d645eef00447-kube-api-access-n2j9z\") pod \"payload-pre-processing-59fbc47bf8-2rh5c\" (UID: \"3425b6dc-5b64-42b6-991d-d645eef00447\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.437212 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.437185 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-scfq4"] Sep 10 14:51:13.440380 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.440366 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.449379 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.449356 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-scfq4"] Sep 10 14:51:13.536650 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.536622 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3425b6dc-5b64-42b6-991d-d645eef00447-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2rh5c\" (UID: \"3425b6dc-5b64-42b6-991d-d645eef00447\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.536650 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.536655 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2j9z\" (UniqueName: \"kubernetes.io/projected/3425b6dc-5b64-42b6-991d-d645eef00447-kube-api-access-n2j9z\") pod \"payload-pre-processing-59fbc47bf8-2rh5c\" (UID: \"3425b6dc-5b64-42b6-991d-d645eef00447\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.536843 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.536676 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92eeed2-5be2-46ad-8c24-65ef2f89b0c8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-scfq4\" (UID: \"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.536843 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.536708 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phjdw\" (UniqueName: \"kubernetes.io/projected/e92eeed2-5be2-46ad-8c24-65ef2f89b0c8-kube-api-access-phjdw\") pod \"payload-processing-5b54b7fbb7-scfq4\" (UID: \"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.537373 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.537354 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3425b6dc-5b64-42b6-991d-d645eef00447-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2rh5c\" (UID: \"3425b6dc-5b64-42b6-991d-d645eef00447\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.552165 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.552133 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2j9z\" (UniqueName: \"kubernetes.io/projected/3425b6dc-5b64-42b6-991d-d645eef00447-kube-api-access-n2j9z\") pod \"payload-pre-processing-59fbc47bf8-2rh5c\" (UID: \"3425b6dc-5b64-42b6-991d-d645eef00447\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.638123 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.638087 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92eeed2-5be2-46ad-8c24-65ef2f89b0c8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-scfq4\" (UID: \"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.638318 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.638133 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phjdw\" (UniqueName: \"kubernetes.io/projected/e92eeed2-5be2-46ad-8c24-65ef2f89b0c8-kube-api-access-phjdw\") pod \"payload-processing-5b54b7fbb7-scfq4\" (UID: \"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.638901 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.638867 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e92eeed2-5be2-46ad-8c24-65ef2f89b0c8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-scfq4\" (UID: \"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.649013 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.648981 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phjdw\" (UniqueName: \"kubernetes.io/projected/e92eeed2-5be2-46ad-8c24-65ef2f89b0c8-kube-api-access-phjdw\") pod \"payload-processing-5b54b7fbb7-scfq4\" (UID: \"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.713551 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.713525 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:13.750086 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.750050 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:13.985310 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:13.985286 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c"] Sep 10 14:51:13.987290 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:51:13.987257 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3425b6dc_5b64_42b6_991d_d645eef00447.slice/crio-ad88fabcdcb6d9b42260bfc3c80114cf8ab8cda95d3a7a3e599e45aaa3755d56 WatchSource:0}: Error finding container ad88fabcdcb6d9b42260bfc3c80114cf8ab8cda95d3a7a3e599e45aaa3755d56: Status 404 returned error can't find the container with id ad88fabcdcb6d9b42260bfc3c80114cf8ab8cda95d3a7a3e599e45aaa3755d56 Sep 10 14:51:14.029727 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:14.029706 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-scfq4"] Sep 10 14:51:14.032730 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:51:14.032705 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92eeed2_5be2_46ad_8c24_65ef2f89b0c8.slice/crio-3f386a0437edf55fbaf5aee8c7d20e08fd9a76331f804d264ee8aa287102d797 WatchSource:0}: Error finding container 3f386a0437edf55fbaf5aee8c7d20e08fd9a76331f804d264ee8aa287102d797: Status 404 returned error can't find the container with id 3f386a0437edf55fbaf5aee8c7d20e08fd9a76331f804d264ee8aa287102d797 Sep 10 14:51:14.209101 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:14.209061 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" event={"ID":"3425b6dc-5b64-42b6-991d-d645eef00447","Type":"ContainerStarted","Data":"ad88fabcdcb6d9b42260bfc3c80114cf8ab8cda95d3a7a3e599e45aaa3755d56"} Sep 10 14:51:14.210331 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:14.210304 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" event={"ID":"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8","Type":"ContainerStarted","Data":"3f386a0437edf55fbaf5aee8c7d20e08fd9a76331f804d264ee8aa287102d797"} Sep 10 14:51:17.229747 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:17.229714 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" event={"ID":"3425b6dc-5b64-42b6-991d-d645eef00447","Type":"ContainerStarted","Data":"65ff688aed21908d30c78f5e7a97d1671eef289240559273f727b599f6f407eb"} Sep 10 14:51:17.230137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:17.229827 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:17.231498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:17.231478 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" event={"ID":"e92eeed2-5be2-46ad-8c24-65ef2f89b0c8","Type":"ContainerStarted","Data":"0907d228dcf8ccb85050eb94a94bff783e027440c93ed1068859a8a0c8b36628"} Sep 10 14:51:17.231620 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:17.231592 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:17.249550 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:17.249476 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" podStartSLOduration=1.124056887 podStartE2EDuration="4.249457338s" totalImagesPullingTime="3.125400451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:51:17.24600053 +0000 UTC m=+533.918026305" watchObservedRunningTime="2026-09-10 14:51:17.249457338 +0000 UTC m=+533.921483112" Sep 10 14:51:17.271223 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:17.270551 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" podStartSLOduration=1.183514571 podStartE2EDuration="4.270529s" totalImagesPullingTime="3.087014429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:51:17.269888567 +0000 UTC m=+533.941914355" watchObservedRunningTime="2026-09-10 14:51:17.270529 +0000 UTC m=+533.942554774" Sep 10 14:51:23.260216 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:23.260170 2584 generic.go:356] "Generic (PLEG): container finished" podID="a1509437-33ca-470a-8b3b-4078375ad640" containerID="b68e645d89bbabc09a0d4b4f49cfb4dcabe8ba639afd68924512115c27578821" exitCode=0 Sep 10 14:51:23.260632 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:23.260232 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" event={"ID":"a1509437-33ca-470a-8b3b-4078375ad640","Type":"ContainerDied","Data":"b68e645d89bbabc09a0d4b4f49cfb4dcabe8ba639afd68924512115c27578821"} Sep 10 14:51:23.260632 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:23.260624 2584 scope.go:117] "RemoveContainer" containerID="b68e645d89bbabc09a0d4b4f49cfb4dcabe8ba639afd68924512115c27578821" Sep 10 14:51:23.261613 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:23.261584 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:51:23.261668 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:23.261652 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:51:23.261701 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:23.261682 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:51:24.269165 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:24.269112 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" event={"ID":"a1509437-33ca-470a-8b3b-4078375ad640","Type":"ContainerStarted","Data":"76262932f11bddc7adffaa3269080c97d6d112bfe1e4f95ddf6316050cccffb0"} Sep 10 14:51:24.672617 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:24.672510 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:51:24.672809 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:24.672656 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rbmhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Sep 10 14:51:24.672809 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:24.672722 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" podUID="a1509437-33ca-470a-8b3b-4078375ad640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Sep 10 14:51:25.672526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:25.672478 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rbmhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Sep 10 14:51:25.673002 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:25.672556 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" podUID="a1509437-33ca-470a-8b3b-4078375ad640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Sep 10 14:51:26.672499 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:26.672458 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rbmhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Sep 10 14:51:26.672724 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:26.672532 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" podUID="a1509437-33ca-470a-8b3b-4078375ad640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Sep 10 14:51:27.672141 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:27.672104 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rbmhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Sep 10 14:51:27.672901 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:27.672419 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" podUID="a1509437-33ca-470a-8b3b-4078375ad640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Sep 10 14:51:27.877837 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:27.877795 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:51:28.238605 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:28.238574 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-scfq4" Sep 10 14:51:28.238775 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:28.238643 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2rh5c" Sep 10 14:51:28.671488 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:28.671392 2584 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rbmhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Sep 10 14:51:28.671736 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:28.671706 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" podUID="a1509437-33ca-470a-8b3b-4078375ad640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Sep 10 14:51:29.674757 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:29.674722 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:51:29.675722 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:51:29.675700 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rbmhd" Sep 10 14:52:23.715874 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:23.715838 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:52:23.716673 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:23.716647 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:52:38.068476 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.068440 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7"] Sep 10 14:52:38.071964 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.071938 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.076306 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.076273 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 10 14:52:38.076306 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.076298 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 10 14:52:38.076492 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.076273 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 10 14:52:38.076608 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.076595 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5tt4z\"" Sep 10 14:52:38.081111 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.081089 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7"] Sep 10 14:52:38.155973 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.155928 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72874\" (UniqueName: \"kubernetes.io/projected/9159abab-61a8-4a2c-8182-dd11ecce2f86-kube-api-access-72874\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.155973 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.155974 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.156251 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.155998 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.156251 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.156093 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9159abab-61a8-4a2c-8182-dd11ecce2f86-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.156251 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.156144 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.156251 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.156232 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.256942 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.256903 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9159abab-61a8-4a2c-8182-dd11ecce2f86-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257142 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.256951 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257142 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.256977 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257142 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.257037 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72874\" (UniqueName: \"kubernetes.io/projected/9159abab-61a8-4a2c-8182-dd11ecce2f86-kube-api-access-72874\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257142 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.257060 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257142 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.257094 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257455 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.257404 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257455 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.257440 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.257523 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.257456 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.259313 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.259292 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9159abab-61a8-4a2c-8182-dd11ecce2f86-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.259572 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.259551 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9159abab-61a8-4a2c-8182-dd11ecce2f86-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.265581 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.265555 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72874\" (UniqueName: \"kubernetes.io/projected/9159abab-61a8-4a2c-8182-dd11ecce2f86-kube-api-access-72874\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7\" (UID: \"9159abab-61a8-4a2c-8182-dd11ecce2f86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.383474 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.383387 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:38.651962 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.651934 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7"] Sep 10 14:52:38.653916 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:52:38.653890 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9159abab_61a8_4a2c_8182_dd11ecce2f86.slice/crio-37893c99109dad13c8594e5326b28392431f9066a6a77f6a8536a3a523da8fd9 WatchSource:0}: Error finding container 37893c99109dad13c8594e5326b28392431f9066a6a77f6a8536a3a523da8fd9: Status 404 returned error can't find the container with id 37893c99109dad13c8594e5326b28392431f9066a6a77f6a8536a3a523da8fd9 Sep 10 14:52:38.655185 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:38.655168 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 14:52:39.608787 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:39.608753 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" event={"ID":"9159abab-61a8-4a2c-8182-dd11ecce2f86","Type":"ContainerStarted","Data":"37893c99109dad13c8594e5326b28392431f9066a6a77f6a8536a3a523da8fd9"} Sep 10 14:52:40.615395 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:40.615356 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" event={"ID":"9159abab-61a8-4a2c-8182-dd11ecce2f86","Type":"ContainerStarted","Data":"efa652aa54799b15eb3f87ecfd5abc102a5dd992d365a4936d070747d4a55a97"} Sep 10 14:52:40.633742 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:40.633687 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" podStartSLOduration=1.649465806 podStartE2EDuration="2.633670734s" totalImagesPullingTime="984.204928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:52:40.632564177 +0000 UTC m=+617.304589948" watchObservedRunningTime="2026-09-10 14:52:40.633670734 +0000 UTC m=+617.305696506" Sep 10 14:52:48.384473 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:48.384393 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:48.384899 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:48.384477 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:48.396359 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:48.396336 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:52:48.662728 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:52:48.662649 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7" Sep 10 14:53:01.971662 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:01.971629 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl"] Sep 10 14:53:01.975609 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:01.975592 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:01.978404 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:01.978385 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 10 14:53:01.986640 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:01.986613 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl"] Sep 10 14:53:02.051505 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.051479 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.051618 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.051514 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzzw2\" (UniqueName: \"kubernetes.io/projected/c999073a-2d10-4a8b-a4df-35dd4bebde68-kube-api-access-lzzw2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.051618 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.051537 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.051618 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.051564 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c999073a-2d10-4a8b-a4df-35dd4bebde68-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.051742 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.051627 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.051742 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.051659 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152378 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152338 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152404 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152441 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152475 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzzw2\" (UniqueName: \"kubernetes.io/projected/c999073a-2d10-4a8b-a4df-35dd4bebde68-kube-api-access-lzzw2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152507 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152530 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c999073a-2d10-4a8b-a4df-35dd4bebde68-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152789 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152751 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152789 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152771 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.152927 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.152904 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.155345 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.155315 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c999073a-2d10-4a8b-a4df-35dd4bebde68-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.155579 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.155561 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c999073a-2d10-4a8b-a4df-35dd4bebde68-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.164028 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.164001 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzzw2\" (UniqueName: \"kubernetes.io/projected/c999073a-2d10-4a8b-a4df-35dd4bebde68-kube-api-access-lzzw2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rrlrl\" (UID: \"c999073a-2d10-4a8b-a4df-35dd4bebde68\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.288367 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.288302 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:02.559232 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.559194 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl"] Sep 10 14:53:02.561573 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:53:02.561549 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc999073a_2d10_4a8b_a4df_35dd4bebde68.slice/crio-d29d6f80300796916b1eb76ddc8735f662f3891070f5ff3680a91fae6c92710b WatchSource:0}: Error finding container d29d6f80300796916b1eb76ddc8735f662f3891070f5ff3680a91fae6c92710b: Status 404 returned error can't find the container with id d29d6f80300796916b1eb76ddc8735f662f3891070f5ff3680a91fae6c92710b Sep 10 14:53:02.718761 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:02.718726 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" event={"ID":"c999073a-2d10-4a8b-a4df-35dd4bebde68","Type":"ContainerStarted","Data":"d29d6f80300796916b1eb76ddc8735f662f3891070f5ff3680a91fae6c92710b"} Sep 10 14:53:03.725913 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:03.725870 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" event={"ID":"c999073a-2d10-4a8b-a4df-35dd4bebde68","Type":"ContainerStarted","Data":"14f4de4aa7db3189f42538badbd2df65d4a8f759cf6335e77c78c767ede124db"} Sep 10 14:53:03.751401 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:03.751315 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" podStartSLOduration=2.561281628 podStartE2EDuration="2.751290281s" totalImagesPullingTime="190.008653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:53:03.748327874 +0000 UTC m=+640.420353659" watchObservedRunningTime="2026-09-10 14:53:03.751290281 +0000 UTC m=+640.423316054" Sep 10 14:53:12.288443 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:12.288400 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:12.288946 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:12.288491 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:12.302044 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:12.302020 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:53:12.779885 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:53:12.779850 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rrlrl" Sep 10 14:55:01.360707 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.360671 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k"] Sep 10 14:55:01.364986 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.364965 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.368245 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.368220 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-74eee9-openshift-default-dockercfg-cghvp\"" Sep 10 14:55:01.385012 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.384985 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k"] Sep 10 14:55:01.434776 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434748 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-data\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.434912 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434782 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-token\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.434912 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434801 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsjz2\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-kube-api-access-zsjz2\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.434912 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434853 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-credential-socket\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.434912 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434868 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8945bb-c09d-4482-9792-3df34af87b4c-istiod-ca-cert\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.434912 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434883 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-podinfo\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.434912 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434905 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-socket\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.435153 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434919 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-certs\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.435153 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.434964 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-envoy\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535537 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535501 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-data\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535545 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-token\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535564 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsjz2\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-kube-api-access-zsjz2\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535599 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-credential-socket\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535624 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8945bb-c09d-4482-9792-3df34af87b4c-istiod-ca-cert\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535646 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-podinfo\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535670 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-socket\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535692 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-certs\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.535717 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-envoy\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.536119 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.535946 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-data\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.536119 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.536006 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-credential-socket\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.536119 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.536065 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-socket\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.536365 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.536342 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8945bb-c09d-4482-9792-3df34af87b4c-istiod-ca-cert\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.536365 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.536353 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-certs\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.538056 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.538029 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-envoy\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.538428 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.538405 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-podinfo\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.545892 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.545869 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-token\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.547559 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.547539 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsjz2\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-kube-api-access-zsjz2\") pod \"e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.675296 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.675191 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:01.957319 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.957281 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k"] Sep 10 14:55:01.962040 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:01.962010 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8945bb_c09d_4482_9792_3df34af87b4c.slice/crio-9d78be735d81a3710d63900b2a3f9f5169baac8c68dde0853a54eadec61b217c WatchSource:0}: Error finding container 9d78be735d81a3710d63900b2a3f9f5169baac8c68dde0853a54eadec61b217c: Status 404 returned error can't find the container with id 9d78be735d81a3710d63900b2a3f9f5169baac8c68dde0853a54eadec61b217c Sep 10 14:55:01.964240 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.964193 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:01.964325 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.964270 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:01.964325 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:01.964300 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:02.246794 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:02.246711 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" event={"ID":"fb8945bb-c09d-4482-9792-3df34af87b4c","Type":"ContainerStarted","Data":"e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a"} Sep 10 14:55:02.246794 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:02.246773 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" event={"ID":"fb8945bb-c09d-4482-9792-3df34af87b4c","Type":"ContainerStarted","Data":"9d78be735d81a3710d63900b2a3f9f5169baac8c68dde0853a54eadec61b217c"} Sep 10 14:55:02.272816 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:02.272771 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podStartSLOduration=1.272756531 podStartE2EDuration="1.272756531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:02.272717733 +0000 UTC m=+758.944743507" watchObservedRunningTime="2026-09-10 14:55:02.272756531 +0000 UTC m=+758.944782303" Sep 10 14:55:02.676250 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:02.676111 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:03.680979 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:03.680946 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:04.255816 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:04.255784 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:04.256806 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:04.256785 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:09.722099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.722050 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9"] Sep 10 14:55:09.726574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.726545 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.729669 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.729640 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3b43a1da-openshift-default-dockercfg-64c85\"" Sep 10 14:55:09.737781 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.737754 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9"] Sep 10 14:55:09.815843 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.815803 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-certs\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816036 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.815964 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-credential-socket\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816036 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.816006 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-envoy\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816152 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.816059 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6f2x\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-kube-api-access-n6f2x\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816152 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.816123 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-token\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816301 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.816230 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-data\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.816287 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-podinfo\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.816333 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/704f62ea-5696-42d6-9975-7e051e95b7d3-istiod-ca-cert\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.816469 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.816394 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-socket\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.917756 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917711 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-data\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.917967 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917768 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-podinfo\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.917967 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917789 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/704f62ea-5696-42d6-9975-7e051e95b7d3-istiod-ca-cert\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.917967 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917833 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-socket\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.917967 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917869 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-certs\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.917967 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917903 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-credential-socket\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.917967 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917935 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-envoy\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.918390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.917989 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6f2x\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-kube-api-access-n6f2x\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.918390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.918038 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-token\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.918390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.918298 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-data\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.918390 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.918374 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-credential-socket\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.918616 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.918432 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-socket\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.918616 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.918472 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-certs\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.918981 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.918957 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/704f62ea-5696-42d6-9975-7e051e95b7d3-istiod-ca-cert\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.920806 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.920769 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-podinfo\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.920806 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.920793 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-envoy\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.926812 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.926766 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-token\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:09.926954 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:09.926927 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6f2x\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-kube-api-access-n6f2x\") pod \"e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:10.040356 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:10.040269 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:10.330103 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:10.330000 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9"] Sep 10 14:55:10.333988 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:10.333950 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704f62ea_5696_42d6_9975_7e051e95b7d3.slice/crio-edb8a32dc07a36836430337beedde8d18c8a3f65bb92bbc0a0f4df320a8e280e WatchSource:0}: Error finding container edb8a32dc07a36836430337beedde8d18c8a3f65bb92bbc0a0f4df320a8e280e: Status 404 returned error can't find the container with id edb8a32dc07a36836430337beedde8d18c8a3f65bb92bbc0a0f4df320a8e280e Sep 10 14:55:10.336303 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:10.336266 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:10.336412 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:10.336337 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:10.336412 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:10.336370 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:11.292955 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:11.292915 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" event={"ID":"704f62ea-5696-42d6-9975-7e051e95b7d3","Type":"ContainerStarted","Data":"4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9"} Sep 10 14:55:11.293381 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:11.292994 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" event={"ID":"704f62ea-5696-42d6-9975-7e051e95b7d3","Type":"ContainerStarted","Data":"edb8a32dc07a36836430337beedde8d18c8a3f65bb92bbc0a0f4df320a8e280e"} Sep 10 14:55:11.316454 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:11.316399 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" podStartSLOduration=2.316379157 podStartE2EDuration="2.316379157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:11.314359915 +0000 UTC m=+767.986385689" watchObservedRunningTime="2026-09-10 14:55:11.316379157 +0000 UTC m=+767.988404928" Sep 10 14:55:12.040527 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:12.040494 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:12.045530 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:12.045506 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:12.297992 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:12.297902 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:12.298941 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:12.298917 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:17.302829 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:17.302792 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9"] Sep 10 14:55:17.303383 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:17.303007 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" podUID="704f62ea-5696-42d6-9975-7e051e95b7d3" containerName="istio-proxy" containerID="cri-o://4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9" gracePeriod=30 Sep 10 14:55:18.523015 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.522985 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v"] Sep 10 14:55:18.526531 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.526512 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.542230 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.542210 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-fbdf5841-gw-openshift-default-dockercfg-h98x6\"" Sep 10 14:55:18.560389 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.560362 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v"] Sep 10 14:55:18.599736 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599709 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-socket\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.599846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599740 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-envoy\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.599846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599763 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-credential-socket\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.599846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599788 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-data\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.599846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599826 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-certs\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.599846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599840 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mbtv\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-kube-api-access-9mbtv\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.600046 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599913 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-podinfo\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.600046 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599950 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-token\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.600046 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.599968 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c989524-35e2-45ca-826c-0db3cabbf8a9-istiod-ca-cert\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701495 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701463 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-token\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701495 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701495 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c989524-35e2-45ca-826c-0db3cabbf8a9-istiod-ca-cert\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701542 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-socket\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701559 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-envoy\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701579 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-credential-socket\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701597 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-data\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701629 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-certs\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701654 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mbtv\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-kube-api-access-9mbtv\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.701739 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.701725 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-podinfo\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.702064 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.702008 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-socket\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.702129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.702110 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-credential-socket\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.702267 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.702238 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-data\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.702396 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.702376 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-certs\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.702443 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.702418 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c989524-35e2-45ca-826c-0db3cabbf8a9-istiod-ca-cert\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.704107 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.704087 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-envoy\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.704351 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.704333 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-podinfo\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.716694 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.716671 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-token\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.716854 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.716837 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mbtv\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-kube-api-access-9mbtv\") pod \"e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:18.837190 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:18.837112 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:19.117792 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.117766 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v"] Sep 10 14:55:19.119355 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:19.119331 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c989524_35e2_45ca_826c_0db3cabbf8a9.slice/crio-02f0659e73468aa26864e59d5c7300f1082f94af8d5ff4bdbc1e0e9d3ff0974f WatchSource:0}: Error finding container 02f0659e73468aa26864e59d5c7300f1082f94af8d5ff4bdbc1e0e9d3ff0974f: Status 404 returned error can't find the container with id 02f0659e73468aa26864e59d5c7300f1082f94af8d5ff4bdbc1e0e9d3ff0974f Sep 10 14:55:19.121572 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.121539 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:19.121633 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.121602 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:19.121669 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.121637 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:19.339078 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.339041 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" event={"ID":"3c989524-35e2-45ca-826c-0db3cabbf8a9","Type":"ContainerStarted","Data":"dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229"} Sep 10 14:55:19.339334 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.339110 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" event={"ID":"3c989524-35e2-45ca-826c-0db3cabbf8a9","Type":"ContainerStarted","Data":"02f0659e73468aa26864e59d5c7300f1082f94af8d5ff4bdbc1e0e9d3ff0974f"} Sep 10 14:55:19.363122 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.363055 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" podStartSLOduration=1.363037875 podStartE2EDuration="1.363037875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:19.362909137 +0000 UTC m=+776.034934911" watchObservedRunningTime="2026-09-10 14:55:19.363037875 +0000 UTC m=+776.035063648" Sep 10 14:55:19.838187 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.838146 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:19.991622 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:19.991588 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v"] Sep 10 14:55:20.344649 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:20.344613 2584 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" secret="" err="secret \"e2e-migrate-fbdf5841-gw-openshift-default-dockercfg-h98x6\" not found" Sep 10 14:55:20.844062 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:20.844027 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:21.349444 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:21.349400 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" podUID="3c989524-35e2-45ca-826c-0db3cabbf8a9" containerName="istio-proxy" containerID="cri-o://dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229" gracePeriod=30 Sep 10 14:55:21.349624 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:21.349525 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:21.350549 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:21.350531 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:22.564525 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.564448 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:22.640130 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640068 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-podinfo\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640130 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640139 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-token\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640426 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640218 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-data\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640426 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640259 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/704f62ea-5696-42d6-9975-7e051e95b7d3-istiod-ca-cert\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640426 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640291 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-envoy\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640426 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640325 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-credential-socket\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640426 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640387 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-socket\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640445 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6f2x\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-kube-api-access-n6f2x\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640492 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-certs\") pod \"704f62ea-5696-42d6-9975-7e051e95b7d3\" (UID: \"704f62ea-5696-42d6-9975-7e051e95b7d3\") " Sep 10 14:55:22.640661 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640545 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:22.640794 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640694 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704f62ea-5696-42d6-9975-7e051e95b7d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:55:22.640888 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640851 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:22.641001 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640975 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:22.641060 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.640995 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:22.641132 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.641114 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.641185 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.641138 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-workload-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.641185 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.641155 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-data\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.641185 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.641169 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/704f62ea-5696-42d6-9975-7e051e95b7d3-istiod-ca-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.641185 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.641182 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-credential-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.642899 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.642873 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:22.643031 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.642946 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-kube-api-access-n6f2x" (OuterVolumeSpecName: "kube-api-access-n6f2x") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "kube-api-access-n6f2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:55:22.643031 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.643006 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 14:55:22.643105 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.643034 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "704f62ea-5696-42d6-9975-7e051e95b7d3" (UID: "704f62ea-5696-42d6-9975-7e051e95b7d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:55:22.741919 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.741882 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6f2x\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-kube-api-access-n6f2x\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.741919 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.741912 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-podinfo\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.741919 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.741927 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-token\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:22.742168 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:22.741936 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/704f62ea-5696-42d6-9975-7e051e95b7d3-istio-envoy\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:23.359947 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.359905 2584 generic.go:356] "Generic (PLEG): container finished" podID="704f62ea-5696-42d6-9975-7e051e95b7d3" containerID="4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9" exitCode=0 Sep 10 14:55:23.360143 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.359953 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" Sep 10 14:55:23.360143 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.359995 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" event={"ID":"704f62ea-5696-42d6-9975-7e051e95b7d3","Type":"ContainerDied","Data":"4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9"} Sep 10 14:55:23.360143 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.360046 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9" event={"ID":"704f62ea-5696-42d6-9975-7e051e95b7d3","Type":"ContainerDied","Data":"edb8a32dc07a36836430337beedde8d18c8a3f65bb92bbc0a0f4df320a8e280e"} Sep 10 14:55:23.360143 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.360068 2584 scope.go:117] "RemoveContainer" containerID="4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9" Sep 10 14:55:23.368996 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.368969 2584 scope.go:117] "RemoveContainer" containerID="4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9" Sep 10 14:55:23.369287 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:55:23.369267 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9\": container with ID starting with 4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9 not found: ID does not exist" containerID="4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9" Sep 10 14:55:23.369361 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.369295 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9"} err="failed to get container status \"4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9\": rpc error: code = NotFound desc = could not find container \"4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9\": container with ID starting with 4dd86946820766032d539a66a1ac32d433dde45e09f7c7e0e96ff8b2fce749f9 not found: ID does not exist" Sep 10 14:55:23.383473 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.383441 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9"] Sep 10 14:55:23.387382 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.387353 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3b43a1da-openshift-default-5997ffb4b6-t8dm9"] Sep 10 14:55:23.763682 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:23.763649 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="704f62ea-5696-42d6-9975-7e051e95b7d3" path="/var/lib/kubelet/pods/704f62ea-5696-42d6-9975-7e051e95b7d3/volumes" Sep 10 14:55:25.496989 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.496947 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv"] Sep 10 14:55:25.497572 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.497551 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704f62ea-5696-42d6-9975-7e051e95b7d3" containerName="istio-proxy" Sep 10 14:55:25.497621 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.497577 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="704f62ea-5696-42d6-9975-7e051e95b7d3" containerName="istio-proxy" Sep 10 14:55:25.497700 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.497686 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="704f62ea-5696-42d6-9975-7e051e95b7d3" containerName="istio-proxy" Sep 10 14:55:25.502751 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.502723 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.506114 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.506083 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-79e15b7c-openshift-default-dockercfg-vss6p\"" Sep 10 14:55:25.515358 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.515327 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv"] Sep 10 14:55:25.574892 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.574856 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-credential-socket\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575097 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.574913 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vkf9\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-kube-api-access-2vkf9\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575097 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.574995 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-istio-token\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575097 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.575056 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-certs\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575097 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.575082 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-envoy\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575341 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.575118 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fecf699-d2be-46ae-8805-ae46dacced96-istiod-ca-cert\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575341 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.575170 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fecf699-d2be-46ae-8805-ae46dacced96-istio-podinfo\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575341 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.575227 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-socket\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.575341 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.575281 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-data\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676172 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676137 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vkf9\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-kube-api-access-2vkf9\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676172 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676175 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-istio-token\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676415 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676310 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-certs\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676415 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-envoy\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676415 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676397 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fecf699-d2be-46ae-8805-ae46dacced96-istiod-ca-cert\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676568 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676434 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fecf699-d2be-46ae-8805-ae46dacced96-istio-podinfo\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676568 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676478 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-socket\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676568 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676540 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-data\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676719 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-credential-socket\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.676916 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676892 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-certs\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.677055 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676945 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-socket\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.677146 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676978 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-data\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.677146 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.676999 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-credential-socket\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.677255 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.677192 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fecf699-d2be-46ae-8805-ae46dacced96-istiod-ca-cert\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.678867 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.678839 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-envoy\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.678955 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.678939 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fecf699-d2be-46ae-8805-ae46dacced96-istio-podinfo\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.688178 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.688153 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-istio-token\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.688999 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.688969 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vkf9\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-kube-api-access-2vkf9\") pod \"e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:25.814055 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:25.813961 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:26.107191 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.107162 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv"] Sep 10 14:55:26.108950 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:26.108916 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fecf699_d2be_46ae_8805_ae46dacced96.slice/crio-df6cb92bac51b25301d578fb8039978e0e86f8e441c635fccb5a5350fbc72dfe WatchSource:0}: Error finding container df6cb92bac51b25301d578fb8039978e0e86f8e441c635fccb5a5350fbc72dfe: Status 404 returned error can't find the container with id df6cb92bac51b25301d578fb8039978e0e86f8e441c635fccb5a5350fbc72dfe Sep 10 14:55:26.111118 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.111083 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:26.111243 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.111169 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:26.111243 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.111224 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:26.377966 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.377928 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" event={"ID":"6fecf699-d2be-46ae-8805-ae46dacced96","Type":"ContainerStarted","Data":"342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c"} Sep 10 14:55:26.378149 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.377998 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" event={"ID":"6fecf699-d2be-46ae-8805-ae46dacced96","Type":"ContainerStarted","Data":"df6cb92bac51b25301d578fb8039978e0e86f8e441c635fccb5a5350fbc72dfe"} Sep 10 14:55:26.408378 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.408314 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" podStartSLOduration=1.408293041 podStartE2EDuration="1.408293041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:26.406942322 +0000 UTC m=+783.078968096" watchObservedRunningTime="2026-09-10 14:55:26.408293041 +0000 UTC m=+783.080318813" Sep 10 14:55:26.589803 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.589775 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:26.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687282 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mbtv\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-kube-api-access-9mbtv\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687329 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-podinfo\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687371 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-envoy\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687399 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c989524-35e2-45ca-826c-0db3cabbf8a9-istiod-ca-cert\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687492 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-token\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687538 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-certs\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.687574 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687566 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-socket\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.688083 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687608 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-data\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.688083 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.687642 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-credential-socket\") pod \"3c989524-35e2-45ca-826c-0db3cabbf8a9\" (UID: \"3c989524-35e2-45ca-826c-0db3cabbf8a9\") " Sep 10 14:55:26.688083 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.688040 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c989524-35e2-45ca-826c-0db3cabbf8a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:55:26.688265 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.688168 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:26.688691 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.688419 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:26.688691 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.688513 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:26.688691 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.688566 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:26.690962 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.690934 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-kube-api-access-9mbtv" (OuterVolumeSpecName: "kube-api-access-9mbtv") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "kube-api-access-9mbtv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:55:26.691110 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.690981 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:26.691222 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.691182 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 14:55:26.691222 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.691122 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c989524-35e2-45ca-826c-0db3cabbf8a9" (UID: "3c989524-35e2-45ca-826c-0db3cabbf8a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:55:26.789092 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789040 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789092 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789082 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-workload-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789092 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789092 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-data\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789092 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789102 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-credential-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789429 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789114 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mbtv\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-kube-api-access-9mbtv\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789429 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789124 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-podinfo\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789429 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789131 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-envoy\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789429 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789139 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c989524-35e2-45ca-826c-0db3cabbf8a9-istiod-ca-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.789429 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.789146 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c989524-35e2-45ca-826c-0db3cabbf8a9-istio-token\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:26.814712 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:26.814680 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:27.384608 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.384576 2584 generic.go:356] "Generic (PLEG): container finished" podID="3c989524-35e2-45ca-826c-0db3cabbf8a9" containerID="dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229" exitCode=0 Sep 10 14:55:27.384805 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.384618 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" Sep 10 14:55:27.384805 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.384678 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" event={"ID":"3c989524-35e2-45ca-826c-0db3cabbf8a9","Type":"ContainerDied","Data":"dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229"} Sep 10 14:55:27.384805 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.384729 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v" event={"ID":"3c989524-35e2-45ca-826c-0db3cabbf8a9","Type":"ContainerDied","Data":"02f0659e73468aa26864e59d5c7300f1082f94af8d5ff4bdbc1e0e9d3ff0974f"} Sep 10 14:55:27.384805 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.384752 2584 scope.go:117] "RemoveContainer" containerID="dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229" Sep 10 14:55:27.399350 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.399323 2584 scope.go:117] "RemoveContainer" containerID="dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229" Sep 10 14:55:27.399712 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:55:27.399692 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229\": container with ID starting with dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229 not found: ID does not exist" containerID="dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229" Sep 10 14:55:27.399754 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.399724 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229"} err="failed to get container status \"dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229\": rpc error: code = NotFound desc = could not find container \"dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229\": container with ID starting with dba7ed538dd4d7a3b34cc99f36af979850d03f4858b30d57f4f69b5acc7d4229 not found: ID does not exist" Sep 10 14:55:27.413324 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.413282 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v"] Sep 10 14:55:27.419294 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.419256 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-fbdf5841-gw-openshift-default-674969c764-ss22v"] Sep 10 14:55:27.763159 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.763122 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c989524-35e2-45ca-826c-0db3cabbf8a9" path="/var/lib/kubelet/pods/3c989524-35e2-45ca-826c-0db3cabbf8a9/volumes" Sep 10 14:55:27.819678 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:27.819648 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:28.390157 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:28.390128 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:28.391245 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:28.391225 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:36.907292 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:36.907261 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv"] Sep 10 14:55:36.907730 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:36.907466 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" podUID="6fecf699-d2be-46ae-8805-ae46dacced96" containerName="istio-proxy" containerID="cri-o://342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c" gracePeriod=30 Sep 10 14:55:40.039570 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.039536 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr"] Sep 10 14:55:40.040021 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.039903 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c989524-35e2-45ca-826c-0db3cabbf8a9" containerName="istio-proxy" Sep 10 14:55:40.040021 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.039915 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c989524-35e2-45ca-826c-0db3cabbf8a9" containerName="istio-proxy" Sep 10 14:55:40.040021 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.039974 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c989524-35e2-45ca-826c-0db3cabbf8a9" containerName="istio-proxy" Sep 10 14:55:40.044357 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.044339 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.049613 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.049583 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-74eee9\"" Sep 10 14:55:40.063096 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.063074 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr"] Sep 10 14:55:40.110578 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.110538 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv5qz\" (UniqueName: \"kubernetes.io/projected/5536ba18-e437-47ac-b2db-d1c0afc87e8f-kube-api-access-bv5qz\") pod \"payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.110742 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.110720 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5536ba18-e437-47ac-b2db-d1c0afc87e8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.211460 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.211424 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5536ba18-e437-47ac-b2db-d1c0afc87e8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.211634 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.211486 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv5qz\" (UniqueName: \"kubernetes.io/projected/5536ba18-e437-47ac-b2db-d1c0afc87e8f-kube-api-access-bv5qz\") pod \"payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.212210 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.212174 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5536ba18-e437-47ac-b2db-d1c0afc87e8f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.237686 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.237658 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv5qz\" (UniqueName: \"kubernetes.io/projected/5536ba18-e437-47ac-b2db-d1c0afc87e8f-kube-api-access-bv5qz\") pod \"payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.354774 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.354705 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:40.624819 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:40.624792 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr"] Sep 10 14:55:40.626271 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:40.626245 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5536ba18_e437_47ac_b2db_d1c0afc87e8f.slice/crio-79b9270cbe8bf58c99d208978961a078bfbddd59f91557d4cfc6d10d9c54f926 WatchSource:0}: Error finding container 79b9270cbe8bf58c99d208978961a078bfbddd59f91557d4cfc6d10d9c54f926: Status 404 returned error can't find the container with id 79b9270cbe8bf58c99d208978961a078bfbddd59f91557d4cfc6d10d9c54f926 Sep 10 14:55:41.455535 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:41.455499 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" event={"ID":"5536ba18-e437-47ac-b2db-d1c0afc87e8f","Type":"ContainerStarted","Data":"07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214"} Sep 10 14:55:41.455886 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:41.455551 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" event={"ID":"5536ba18-e437-47ac-b2db-d1c0afc87e8f","Type":"ContainerStarted","Data":"79b9270cbe8bf58c99d208978961a078bfbddd59f91557d4cfc6d10d9c54f926"} Sep 10 14:55:41.455886 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:41.455618 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:41.481944 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:41.481899 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" podStartSLOduration=1.481887632 podStartE2EDuration="1.481887632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:41.481532777 +0000 UTC m=+798.153558548" watchObservedRunningTime="2026-09-10 14:55:41.481887632 +0000 UTC m=+798.153913404" Sep 10 14:55:42.168188 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.168130 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:42.233425 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233394 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-data\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233571 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233468 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-certs\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233571 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233492 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-envoy\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233571 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233529 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fecf699-d2be-46ae-8805-ae46dacced96-istio-podinfo\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233581 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-socket\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233627 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fecf699-d2be-46ae-8805-ae46dacced96-istiod-ca-cert\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233626 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-data" (OuterVolumeSpecName: "istio-data") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:42.233732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233659 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vkf9\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-kube-api-access-2vkf9\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233732 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233705 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-credential-socket\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233973 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233728 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:42.233973 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233743 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-istio-token\") pod \"6fecf699-d2be-46ae-8805-ae46dacced96\" (UID: \"6fecf699-d2be-46ae-8805-ae46dacced96\") " Sep 10 14:55:42.233973 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233887 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:42.234129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.233998 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:42.234129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.234028 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fecf699-d2be-46ae-8805-ae46dacced96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:55:42.234245 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.234218 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-data\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.234245 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.234238 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.234350 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.234252 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-workload-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.234350 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.234261 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fecf699-d2be-46ae-8805-ae46dacced96-istiod-ca-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.234350 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.234270 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-credential-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.235835 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.235804 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-istio-token" (OuterVolumeSpecName: "istio-token") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:55:42.235951 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.235844 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-kube-api-access-2vkf9" (OuterVolumeSpecName: "kube-api-access-2vkf9") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "kube-api-access-2vkf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:55:42.236076 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.236056 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fecf699-d2be-46ae-8805-ae46dacced96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 14:55:42.236273 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.236246 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6fecf699-d2be-46ae-8805-ae46dacced96" (UID: "6fecf699-d2be-46ae-8805-ae46dacced96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:55:42.335171 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.335149 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vkf9\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-kube-api-access-2vkf9\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.335171 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.335169 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fecf699-d2be-46ae-8805-ae46dacced96-istio-token\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.335333 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.335178 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fecf699-d2be-46ae-8805-ae46dacced96-istio-envoy\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.335333 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.335186 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fecf699-d2be-46ae-8805-ae46dacced96-istio-podinfo\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:55:42.461983 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.461950 2584 generic.go:356] "Generic (PLEG): container finished" podID="6fecf699-d2be-46ae-8805-ae46dacced96" containerID="342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c" exitCode=0 Sep 10 14:55:42.462402 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.461990 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" Sep 10 14:55:42.462402 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.462040 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" event={"ID":"6fecf699-d2be-46ae-8805-ae46dacced96","Type":"ContainerDied","Data":"342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c"} Sep 10 14:55:42.462402 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.462091 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv" event={"ID":"6fecf699-d2be-46ae-8805-ae46dacced96","Type":"ContainerDied","Data":"df6cb92bac51b25301d578fb8039978e0e86f8e441c635fccb5a5350fbc72dfe"} Sep 10 14:55:42.462402 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.462114 2584 scope.go:117] "RemoveContainer" containerID="342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c" Sep 10 14:55:42.471145 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.471129 2584 scope.go:117] "RemoveContainer" containerID="342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c" Sep 10 14:55:42.471428 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:55:42.471408 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c\": container with ID starting with 342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c not found: ID does not exist" containerID="342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c" Sep 10 14:55:42.471479 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.471435 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c"} err="failed to get container status \"342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c\": rpc error: code = NotFound desc = could not find container \"342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c\": container with ID starting with 342c13919962fa1afd718cc9a7f386470d7fd98ce7c1c3984f879053d56feb2c not found: ID does not exist" Sep 10 14:55:42.494947 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.494926 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv"] Sep 10 14:55:42.501076 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:42.501057 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-79e15b7c-openshift-default-7d4f64d5b-vhjfv"] Sep 10 14:55:43.764179 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:43.764142 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fecf699-d2be-46ae-8805-ae46dacced96" path="/var/lib/kubelet/pods/6fecf699-d2be-46ae-8805-ae46dacced96/volumes" Sep 10 14:55:45.209891 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.209856 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2"] Sep 10 14:55:45.210369 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.210247 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fecf699-d2be-46ae-8805-ae46dacced96" containerName="istio-proxy" Sep 10 14:55:45.210369 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.210260 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fecf699-d2be-46ae-8805-ae46dacced96" containerName="istio-proxy" Sep 10 14:55:45.210369 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.210313 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fecf699-d2be-46ae-8805-ae46dacced96" containerName="istio-proxy" Sep 10 14:55:45.214378 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.214359 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.217164 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.217138 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ff661a-openshift-default-dockercfg-pjtjv\"" Sep 10 14:55:45.225871 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.225847 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2"] Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271689 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-token\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-podinfo\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271784 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-credential-socket\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271810 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-data\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271835 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnm9v\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-kube-api-access-jnm9v\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271882 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-socket\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-certs\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271948 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-envoy\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.272332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.271988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istiod-ca-cert\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373486 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373447 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-podinfo\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373501 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-credential-socket\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373534 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-data\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373561 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnm9v\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-kube-api-access-jnm9v\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373616 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-socket\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373667 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373655 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-certs\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373955 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373684 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-envoy\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373955 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istiod-ca-cert\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.373955 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.373824 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-token\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.375047 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.375014 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-data\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.375446 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.375129 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-socket\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.375541 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.375364 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-certs\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.375864 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.375834 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istiod-ca-cert\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.376130 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.376107 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-credential-socket\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.377694 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.377668 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-podinfo\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.378253 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.378232 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-envoy\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.382904 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.382875 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-token\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.383756 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.383739 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnm9v\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-kube-api-access-jnm9v\") pod \"e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.527170 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.527102 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:45.839846 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.839821 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2"] Sep 10 14:55:45.841187 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:45.841153 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6cfd0a_9a97_4182_874b_7ccb1e139db0.slice/crio-1cb69a297ac6bf97a0002c444e507574dbc1b4b7daaaf3e37394714ce627e2de WatchSource:0}: Error finding container 1cb69a297ac6bf97a0002c444e507574dbc1b4b7daaaf3e37394714ce627e2de: Status 404 returned error can't find the container with id 1cb69a297ac6bf97a0002c444e507574dbc1b4b7daaaf3e37394714ce627e2de Sep 10 14:55:45.843523 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.843487 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:45.843609 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.843556 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:45.843609 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:45.843585 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:46.486301 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:46.486225 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" event={"ID":"7c6cfd0a-9a97-4182-874b-7ccb1e139db0","Type":"ContainerStarted","Data":"85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1"} Sep 10 14:55:46.486301 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:46.486279 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" event={"ID":"7c6cfd0a-9a97-4182-874b-7ccb1e139db0","Type":"ContainerStarted","Data":"1cb69a297ac6bf97a0002c444e507574dbc1b4b7daaaf3e37394714ce627e2de"} Sep 10 14:55:46.528071 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:46.528048 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:46.530730 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:46.530669 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" podStartSLOduration=1.5306426100000001 podStartE2EDuration="1.53064261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:46.52932199 +0000 UTC m=+803.201347775" watchObservedRunningTime="2026-09-10 14:55:46.53064261 +0000 UTC m=+803.202668383" Sep 10 14:55:46.533481 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:46.533458 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:47.491676 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:47.491647 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:47.492825 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:47.492801 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:55:50.546795 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.546755 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff"] Sep 10 14:55:50.550502 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.550480 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.553622 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.553599 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ff661a\"" Sep 10 14:55:50.553756 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.553713 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ff661a-dockercfg-st4lz\"" Sep 10 14:55:50.561325 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.561302 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff"] Sep 10 14:55:50.597389 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.597357 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h"] Sep 10 14:55:50.600743 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.600727 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:50.616233 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.616191 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h"] Sep 10 14:55:50.626155 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.626127 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpdz2\" (UniqueName: \"kubernetes.io/projected/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-kube-api-access-rpdz2\") pod \"payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.626425 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.626398 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.727444 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.727411 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpdz2\" (UniqueName: \"kubernetes.io/projected/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-kube-api-access-rpdz2\") pod \"payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.727444 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.727452 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsvfh\" (UniqueName: \"kubernetes.io/projected/cf06af0e-43a1-4d68-83dc-b24eff544b3c-kube-api-access-jsvfh\") pod \"payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:50.727695 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.727586 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.727695 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.727682 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf06af0e-43a1-4d68-83dc-b24eff544b3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:50.728369 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.728349 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.736290 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.736264 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpdz2\" (UniqueName: \"kubernetes.io/projected/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-kube-api-access-rpdz2\") pod \"payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.829192 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.829083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsvfh\" (UniqueName: \"kubernetes.io/projected/cf06af0e-43a1-4d68-83dc-b24eff544b3c-kube-api-access-jsvfh\") pod \"payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:50.829395 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.829281 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf06af0e-43a1-4d68-83dc-b24eff544b3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:50.829985 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.829964 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf06af0e-43a1-4d68-83dc-b24eff544b3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:50.839358 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.839334 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsvfh\" (UniqueName: \"kubernetes.io/projected/cf06af0e-43a1-4d68-83dc-b24eff544b3c-kube-api-access-jsvfh\") pod \"payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:50.862247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.862191 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:50.911905 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:50.911569 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:51.136292 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.136269 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff"] Sep 10 14:55:51.137788 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:51.137761 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca3c631_fc6a_46b2_9f40_5910fe677a7e.slice/crio-b68061ad5f553869873fdff98e53dc52c6894edd411cd6f766821ead4a4a392f WatchSource:0}: Error finding container b68061ad5f553869873fdff98e53dc52c6894edd411cd6f766821ead4a4a392f: Status 404 returned error can't find the container with id b68061ad5f553869873fdff98e53dc52c6894edd411cd6f766821ead4a4a392f Sep 10 14:55:51.201557 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.201530 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h"] Sep 10 14:55:51.202815 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:55:51.202787 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf06af0e_43a1_4d68_83dc_b24eff544b3c.slice/crio-4b4123920cd398644287f92329053f46dbcbc12005f853a3e0e022911342856d WatchSource:0}: Error finding container 4b4123920cd398644287f92329053f46dbcbc12005f853a3e0e022911342856d: Status 404 returned error can't find the container with id 4b4123920cd398644287f92329053f46dbcbc12005f853a3e0e022911342856d Sep 10 14:55:51.512137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.512079 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" event={"ID":"9ca3c631-fc6a-46b2-9f40-5910fe677a7e","Type":"ContainerStarted","Data":"f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188"} Sep 10 14:55:51.512137 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.512142 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:55:51.512377 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.512162 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" event={"ID":"9ca3c631-fc6a-46b2-9f40-5910fe677a7e","Type":"ContainerStarted","Data":"b68061ad5f553869873fdff98e53dc52c6894edd411cd6f766821ead4a4a392f"} Sep 10 14:55:51.513766 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.513730 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" event={"ID":"cf06af0e-43a1-4d68-83dc-b24eff544b3c","Type":"ContainerStarted","Data":"a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6"} Sep 10 14:55:51.513876 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.513784 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" event={"ID":"cf06af0e-43a1-4d68-83dc-b24eff544b3c","Type":"ContainerStarted","Data":"4b4123920cd398644287f92329053f46dbcbc12005f853a3e0e022911342856d"} Sep 10 14:55:51.513876 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.513812 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:55:51.529440 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.529383 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" podStartSLOduration=1.529365419 podStartE2EDuration="1.529365419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:51.528356203 +0000 UTC m=+808.200381977" watchObservedRunningTime="2026-09-10 14:55:51.529365419 +0000 UTC m=+808.201391192" Sep 10 14:55:51.546836 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:51.546796 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" podStartSLOduration=1.546784128 podStartE2EDuration="1.546784128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:51.544799925 +0000 UTC m=+808.216825698" watchObservedRunningTime="2026-09-10 14:55:51.546784128 +0000 UTC m=+808.218809900" Sep 10 14:55:52.465000 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:52.464956 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:55:53.526098 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:53.526060 2584 generic.go:356] "Generic (PLEG): container finished" podID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerID="e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a" exitCode=0 Sep 10 14:55:53.526510 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:53.526132 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" event={"ID":"fb8945bb-c09d-4482-9792-3df34af87b4c","Type":"ContainerDied","Data":"e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a"} Sep 10 14:55:53.526563 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:53.526515 2584 scope.go:117] "RemoveContainer" containerID="e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a" Sep 10 14:55:53.527498 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:53.527471 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:53.527558 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:53.527535 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:53.527597 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:53.527564 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:55:54.533487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:54.533458 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" event={"ID":"fb8945bb-c09d-4482-9792-3df34af87b4c","Type":"ContainerStarted","Data":"6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0"} Sep 10 14:55:54.675560 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:54.675526 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:55:54.675795 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:54.675771 2584 patch_prober.go:28] interesting pod/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Sep 10 14:55:54.675880 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:54.675825 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Sep 10 14:55:55.676024 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:55.675987 2584 patch_prober.go:28] interesting pod/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Sep 10 14:55:55.676455 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:55.676042 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Sep 10 14:55:56.675736 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:56.675697 2584 patch_prober.go:28] interesting pod/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Sep 10 14:55:56.675894 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:56.675764 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Sep 10 14:55:57.676319 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:57.676276 2584 patch_prober.go:28] interesting pod/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Sep 10 14:55:57.676706 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:57.676353 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Sep 10 14:55:58.675577 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:58.675539 2584 patch_prober.go:28] interesting pod/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Sep 10 14:55:58.675752 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:58.675643 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Sep 10 14:55:59.678099 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:59.678057 2584 patch_prober.go:28] interesting pod/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Sep 10 14:55:59.678722 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:55:59.678674 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Sep 10 14:56:00.679727 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:00.679678 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:56:00.680129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:00.680019 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:56:00.680761 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:00.680739 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:56:02.522881 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:02.522842 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:56:02.523295 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:02.522919 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:56:08.619071 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:08.619026 2584 generic.go:356] "Generic (PLEG): container finished" podID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerID="85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1" exitCode=0 Sep 10 14:56:08.619874 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:08.619102 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" event={"ID":"7c6cfd0a-9a97-4182-874b-7ccb1e139db0","Type":"ContainerDied","Data":"85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1"} Sep 10 14:56:08.619874 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:08.619657 2584 scope.go:117] "RemoveContainer" containerID="85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1" Sep 10 14:56:08.621326 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:08.621291 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:56:08.621400 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:08.621385 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:56:08.621437 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:08.621422 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:56:09.626238 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:09.626180 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" event={"ID":"7c6cfd0a-9a97-4182-874b-7ccb1e139db0","Type":"ContainerStarted","Data":"043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8"} Sep 10 14:56:10.527594 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:10.527560 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:56:10.527827 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:10.527807 2584 patch_prober.go:28] interesting pod/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Sep 10 14:56:10.527878 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:10.527852 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Sep 10 14:56:11.527707 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:11.527666 2584 patch_prober.go:28] interesting pod/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Sep 10 14:56:11.528082 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:11.527725 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Sep 10 14:56:12.527545 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:12.527512 2584 patch_prober.go:28] interesting pod/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Sep 10 14:56:12.527714 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:12.527599 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Sep 10 14:56:13.528219 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:13.528162 2584 patch_prober.go:28] interesting pod/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Sep 10 14:56:13.528607 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:13.528247 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Sep 10 14:56:14.527645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:14.527604 2584 patch_prober.go:28] interesting pod/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Sep 10 14:56:14.527824 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:14.527661 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Sep 10 14:56:15.532169 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:15.532131 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:56:15.532649 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:15.532445 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:56:15.533117 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:56:15.533099 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:57:23.745487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:57:23.745411 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:57:23.747918 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:57:23.747895 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 14:59:07.576406 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:07.576369 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr"] Sep 10 14:59:07.577345 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:07.577193 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" podUID="5536ba18-e437-47ac-b2db-d1c0afc87e8f" containerName="payload-pre-processing" containerID="cri-o://07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214" gracePeriod=30 Sep 10 14:59:07.857083 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:07.857056 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:59:07.943844 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:07.943803 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5536ba18-e437-47ac-b2db-d1c0afc87e8f-plugins-config-volume\") pod \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " Sep 10 14:59:07.944054 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:07.943886 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv5qz\" (UniqueName: \"kubernetes.io/projected/5536ba18-e437-47ac-b2db-d1c0afc87e8f-kube-api-access-bv5qz\") pod \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\" (UID: \"5536ba18-e437-47ac-b2db-d1c0afc87e8f\") " Sep 10 14:59:07.944234 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:07.944179 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5536ba18-e437-47ac-b2db-d1c0afc87e8f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5536ba18-e437-47ac-b2db-d1c0afc87e8f" (UID: "5536ba18-e437-47ac-b2db-d1c0afc87e8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:59:07.946043 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:07.946011 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5536ba18-e437-47ac-b2db-d1c0afc87e8f-kube-api-access-bv5qz" (OuterVolumeSpecName: "kube-api-access-bv5qz") pod "5536ba18-e437-47ac-b2db-d1c0afc87e8f" (UID: "5536ba18-e437-47ac-b2db-d1c0afc87e8f"). InnerVolumeSpecName "kube-api-access-bv5qz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:59:08.044847 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.044801 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv5qz\" (UniqueName: \"kubernetes.io/projected/5536ba18-e437-47ac-b2db-d1c0afc87e8f-kube-api-access-bv5qz\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:08.044847 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.044838 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5536ba18-e437-47ac-b2db-d1c0afc87e8f-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:08.485666 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.485630 2584 generic.go:356] "Generic (PLEG): container finished" podID="5536ba18-e437-47ac-b2db-d1c0afc87e8f" containerID="07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214" exitCode=0 Sep 10 14:59:08.485932 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.485677 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" Sep 10 14:59:08.485932 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.485723 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" event={"ID":"5536ba18-e437-47ac-b2db-d1c0afc87e8f","Type":"ContainerDied","Data":"07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214"} Sep 10 14:59:08.485932 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.485774 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr" event={"ID":"5536ba18-e437-47ac-b2db-d1c0afc87e8f","Type":"ContainerDied","Data":"79b9270cbe8bf58c99d208978961a078bfbddd59f91557d4cfc6d10d9c54f926"} Sep 10 14:59:08.485932 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.485795 2584 scope.go:117] "RemoveContainer" containerID="07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214" Sep 10 14:59:08.494744 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.494721 2584 scope.go:117] "RemoveContainer" containerID="07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214" Sep 10 14:59:08.495038 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:59:08.495019 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214\": container with ID starting with 07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214 not found: ID does not exist" containerID="07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214" Sep 10 14:59:08.495085 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.495047 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214"} err="failed to get container status \"07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214\": rpc error: code = NotFound desc = could not find container \"07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214\": container with ID starting with 07ae788f30b4307459094ef1617b5b2f7e98f9c0ffbf7a1e69f63caad0af3214 not found: ID does not exist" Sep 10 14:59:08.515251 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.515192 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr"] Sep 10 14:59:08.523134 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:08.523100 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-74eee9-55b5557bb6-52dfr"] Sep 10 14:59:09.766420 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:09.764861 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5536ba18-e437-47ac-b2db-d1c0afc87e8f" path="/var/lib/kubelet/pods/5536ba18-e437-47ac-b2db-d1c0afc87e8f/volumes" Sep 10 14:59:15.006601 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.006569 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k"] Sep 10 14:59:15.007009 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.006781 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" containerID="cri-o://6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0" gracePeriod=30 Sep 10 14:59:15.071642 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.071611 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5"] Sep 10 14:59:15.071973 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.071960 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5536ba18-e437-47ac-b2db-d1c0afc87e8f" containerName="payload-pre-processing" Sep 10 14:59:15.072017 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.071974 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5536ba18-e437-47ac-b2db-d1c0afc87e8f" containerName="payload-pre-processing" Sep 10 14:59:15.072051 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.072038 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5536ba18-e437-47ac-b2db-d1c0afc87e8f" containerName="payload-pre-processing" Sep 10 14:59:15.076515 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.076490 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.079422 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.079400 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-5899e8\"" Sep 10 14:59:15.088168 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.088145 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5"] Sep 10 14:59:15.112012 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.111987 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l49r2\" (UniqueName: \"kubernetes.io/projected/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-kube-api-access-l49r2\") pod \"payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.112111 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.112082 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.201476 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.201440 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln"] Sep 10 14:59:15.205618 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.205589 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.209682 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.209658 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-5899e8-dockercfg-h8z7f\"" Sep 10 14:59:15.214227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.212954 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l49r2\" (UniqueName: \"kubernetes.io/projected/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-kube-api-access-l49r2\") pod \"payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.214227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.213073 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.214227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.213934 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.215335 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.215313 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln"] Sep 10 14:59:15.227884 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.227857 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l49r2\" (UniqueName: \"kubernetes.io/projected/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-kube-api-access-l49r2\") pod \"payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.314662 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.314585 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl9kw\" (UniqueName: \"kubernetes.io/projected/ccc11795-fc2f-4b37-be41-052d2abae239-kube-api-access-hl9kw\") pod \"payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.314662 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.314631 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc11795-fc2f-4b37-be41-052d2abae239-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.387247 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.387218 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:15.415943 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.415909 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9kw\" (UniqueName: \"kubernetes.io/projected/ccc11795-fc2f-4b37-be41-052d2abae239-kube-api-access-hl9kw\") pod \"payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.416055 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.415970 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc11795-fc2f-4b37-be41-052d2abae239-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.416689 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.416671 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc11795-fc2f-4b37-be41-052d2abae239-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.424332 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.424311 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl9kw\" (UniqueName: \"kubernetes.io/projected/ccc11795-fc2f-4b37-be41-052d2abae239-kube-api-access-hl9kw\") pod \"payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.526065 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.526035 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:15.660951 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.660923 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5"] Sep 10 14:59:15.662697 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:59:15.662674 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12336108_5b25_4a87_b4a3_f2f65d9c3c4b.slice/crio-cf6cc030f1bc594372586d0e698edd837f271642e0443e3a65380620401b158d WatchSource:0}: Error finding container cf6cc030f1bc594372586d0e698edd837f271642e0443e3a65380620401b158d: Status 404 returned error can't find the container with id cf6cc030f1bc594372586d0e698edd837f271642e0443e3a65380620401b158d Sep 10 14:59:15.683394 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.683367 2584 patch_prober.go:28] interesting pod/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 14:59:15.683515 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.683426 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 14:59:15.800850 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:15.800818 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln"] Sep 10 14:59:15.801111 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:59:15.801090 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc11795_fc2f_4b37_be41_052d2abae239.slice/crio-7497d8d1cbd93e64e2ed89f364c88d53cb5ab1cd8e03ba8b64e77e9c8c4484a5 WatchSource:0}: Error finding container 7497d8d1cbd93e64e2ed89f364c88d53cb5ab1cd8e03ba8b64e77e9c8c4484a5: Status 404 returned error can't find the container with id 7497d8d1cbd93e64e2ed89f364c88d53cb5ab1cd8e03ba8b64e77e9c8c4484a5 Sep 10 14:59:16.532492 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.532452 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" event={"ID":"12336108-5b25-4a87-b4a3-f2f65d9c3c4b","Type":"ContainerStarted","Data":"a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852"} Sep 10 14:59:16.532916 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.532554 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:16.532916 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.532578 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" event={"ID":"12336108-5b25-4a87-b4a3-f2f65d9c3c4b","Type":"ContainerStarted","Data":"cf6cc030f1bc594372586d0e698edd837f271642e0443e3a65380620401b158d"} Sep 10 14:59:16.534008 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.533982 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" event={"ID":"ccc11795-fc2f-4b37-be41-052d2abae239","Type":"ContainerStarted","Data":"e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c"} Sep 10 14:59:16.534195 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.534022 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" event={"ID":"ccc11795-fc2f-4b37-be41-052d2abae239","Type":"ContainerStarted","Data":"7497d8d1cbd93e64e2ed89f364c88d53cb5ab1cd8e03ba8b64e77e9c8c4484a5"} Sep 10 14:59:16.534195 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.534154 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:16.555176 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.555126 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" podStartSLOduration=1.555114341 podStartE2EDuration="1.555114341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:16.553005056 +0000 UTC m=+1013.225030829" watchObservedRunningTime="2026-09-10 14:59:16.555114341 +0000 UTC m=+1013.227140143" Sep 10 14:59:16.574067 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:16.574024 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" podStartSLOduration=1.574007544 podStartE2EDuration="1.574007544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:16.571530827 +0000 UTC m=+1013.243556594" watchObservedRunningTime="2026-09-10 14:59:16.574007544 +0000 UTC m=+1013.246033319" Sep 10 14:59:20.253129 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.253105 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:59:20.356421 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356324 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-token\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356421 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356368 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsjz2\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-kube-api-access-zsjz2\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356421 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356391 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-certs\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356421 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356421 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-podinfo\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356752 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356459 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8945bb-c09d-4482-9792-3df34af87b4c-istiod-ca-cert\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356752 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356489 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-data\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356752 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356586 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-envoy\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356752 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356623 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-socket\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.356752 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356672 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-credential-socket\") pod \"fb8945bb-c09d-4482-9792-3df34af87b4c\" (UID: \"fb8945bb-c09d-4482-9792-3df34af87b4c\") " Sep 10 14:59:20.357008 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356717 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:20.357008 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356876 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:20.357008 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356908 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb8945bb-c09d-4482-9792-3df34af87b4c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:59:20.357008 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.356996 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:20.357329 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.357089 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb8945bb-c09d-4482-9792-3df34af87b4c-istiod-ca-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.357329 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.357100 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-data\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.357329 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.357108 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-credential-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.357329 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.357109 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:20.357329 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.357125 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.359047 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.359014 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:20.359132 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.359025 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:59:20.359319 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.359282 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-kube-api-access-zsjz2" (OuterVolumeSpecName: "kube-api-access-zsjz2") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "kube-api-access-zsjz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:59:20.359424 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.359350 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb8945bb-c09d-4482-9792-3df34af87b4c" (UID: "fb8945bb-c09d-4482-9792-3df34af87b4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 14:59:20.458045 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.458007 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-envoy\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.458045 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.458037 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb8945bb-c09d-4482-9792-3df34af87b4c-workload-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.458045 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.458048 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-token\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.458301 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.458057 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsjz2\" (UniqueName: \"kubernetes.io/projected/fb8945bb-c09d-4482-9792-3df34af87b4c-kube-api-access-zsjz2\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.458301 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.458076 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb8945bb-c09d-4482-9792-3df34af87b4c-istio-podinfo\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:20.558828 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.558796 2584 generic.go:356] "Generic (PLEG): container finished" podID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerID="6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0" exitCode=0 Sep 10 14:59:20.558979 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.558843 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" Sep 10 14:59:20.558979 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.558877 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" event={"ID":"fb8945bb-c09d-4482-9792-3df34af87b4c","Type":"ContainerDied","Data":"6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0"} Sep 10 14:59:20.558979 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.558929 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k" event={"ID":"fb8945bb-c09d-4482-9792-3df34af87b4c","Type":"ContainerDied","Data":"9d78be735d81a3710d63900b2a3f9f5169baac8c68dde0853a54eadec61b217c"} Sep 10 14:59:20.558979 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.558951 2584 scope.go:117] "RemoveContainer" containerID="6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0" Sep 10 14:59:20.568408 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.568390 2584 scope.go:117] "RemoveContainer" containerID="e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a" Sep 10 14:59:20.582740 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.582707 2584 scope.go:117] "RemoveContainer" containerID="6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0" Sep 10 14:59:20.583039 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.583000 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k"] Sep 10 14:59:20.583163 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:59:20.583145 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0\": container with ID starting with 6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0 not found: ID does not exist" containerID="6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0" Sep 10 14:59:20.583233 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.583171 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0"} err="failed to get container status \"6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0\": rpc error: code = NotFound desc = could not find container \"6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0\": container with ID starting with 6e15e566405c304fb0b1d2e23ee6b71f8ffe367119b502975592defd695aebb0 not found: ID does not exist" Sep 10 14:59:20.583233 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.583189 2584 scope.go:117] "RemoveContainer" containerID="e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a" Sep 10 14:59:20.583502 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:59:20.583480 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a\": container with ID starting with e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a not found: ID does not exist" containerID="e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a" Sep 10 14:59:20.583543 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.583510 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a"} err="failed to get container status \"e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a\": rpc error: code = NotFound desc = could not find container \"e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a\": container with ID starting with e5063bd1de8e450dbd7d7d1f05bbed2ac2c161c3c00e34c8a802802fa3b6ac8a not found: ID does not exist" Sep 10 14:59:20.589297 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:20.589273 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-74eee9-openshift-default-77dcfcc64d-svz6k"] Sep 10 14:59:21.763469 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:21.763423 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" path="/var/lib/kubelet/pods/fb8945bb-c09d-4482-9792-3df34af87b4c/volumes" Sep 10 14:59:26.221244 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.221187 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff"] Sep 10 14:59:26.221731 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.221425 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" podUID="9ca3c631-fc6a-46b2-9f40-5910fe677a7e" containerName="payload-processing" containerID="cri-o://f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188" gracePeriod=30 Sep 10 14:59:26.230713 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.230681 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h"] Sep 10 14:59:26.230987 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.230935 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" podUID="cf06af0e-43a1-4d68-83dc-b24eff544b3c" containerName="payload-pre-processing" containerID="cri-o://a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6" gracePeriod=30 Sep 10 14:59:26.506737 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.506706 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:59:26.509645 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.509618 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:59:26.593272 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.593235 2584 generic.go:356] "Generic (PLEG): container finished" podID="9ca3c631-fc6a-46b2-9f40-5910fe677a7e" containerID="f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188" exitCode=0 Sep 10 14:59:26.593272 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.593267 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" Sep 10 14:59:26.593555 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.593319 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" event={"ID":"9ca3c631-fc6a-46b2-9f40-5910fe677a7e","Type":"ContainerDied","Data":"f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188"} Sep 10 14:59:26.593555 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.593412 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff" event={"ID":"9ca3c631-fc6a-46b2-9f40-5910fe677a7e","Type":"ContainerDied","Data":"b68061ad5f553869873fdff98e53dc52c6894edd411cd6f766821ead4a4a392f"} Sep 10 14:59:26.593555 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.593445 2584 scope.go:117] "RemoveContainer" containerID="f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188" Sep 10 14:59:26.597883 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.597859 2584 generic.go:356] "Generic (PLEG): container finished" podID="cf06af0e-43a1-4d68-83dc-b24eff544b3c" containerID="a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6" exitCode=0 Sep 10 14:59:26.597993 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.597903 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" event={"ID":"cf06af0e-43a1-4d68-83dc-b24eff544b3c","Type":"ContainerDied","Data":"a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6"} Sep 10 14:59:26.597993 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.597951 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" event={"ID":"cf06af0e-43a1-4d68-83dc-b24eff544b3c","Type":"ContainerDied","Data":"4b4123920cd398644287f92329053f46dbcbc12005f853a3e0e022911342856d"} Sep 10 14:59:26.597993 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.597911 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h" Sep 10 14:59:26.605653 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.605630 2584 scope.go:117] "RemoveContainer" containerID="f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188" Sep 10 14:59:26.606031 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:59:26.606009 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188\": container with ID starting with f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188 not found: ID does not exist" containerID="f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188" Sep 10 14:59:26.606086 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.606043 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188"} err="failed to get container status \"f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188\": rpc error: code = NotFound desc = could not find container \"f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188\": container with ID starting with f35e4d21c5b08420efa54e225d9bdb6423dec3687779ec06cbb591ef50255188 not found: ID does not exist" Sep 10 14:59:26.606086 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.606061 2584 scope.go:117] "RemoveContainer" containerID="a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6" Sep 10 14:59:26.614526 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.614502 2584 scope.go:117] "RemoveContainer" containerID="a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6" Sep 10 14:59:26.614855 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:59:26.614835 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6\": container with ID starting with a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6 not found: ID does not exist" containerID="a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6" Sep 10 14:59:26.614935 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.614864 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6"} err="failed to get container status \"a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6\": rpc error: code = NotFound desc = could not find container \"a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6\": container with ID starting with a28c2209258221bfa972c9bc796608651aa4063c9a367f46f629a6250f1a01b6 not found: ID does not exist" Sep 10 14:59:26.620215 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.620179 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpdz2\" (UniqueName: \"kubernetes.io/projected/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-kube-api-access-rpdz2\") pod \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " Sep 10 14:59:26.620303 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.620254 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf06af0e-43a1-4d68-83dc-b24eff544b3c-plugins-config-volume\") pod \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " Sep 10 14:59:26.620303 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.620280 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsvfh\" (UniqueName: \"kubernetes.io/projected/cf06af0e-43a1-4d68-83dc-b24eff544b3c-kube-api-access-jsvfh\") pod \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\" (UID: \"cf06af0e-43a1-4d68-83dc-b24eff544b3c\") " Sep 10 14:59:26.620391 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.620316 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-plugins-config-volume\") pod \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\" (UID: \"9ca3c631-fc6a-46b2-9f40-5910fe677a7e\") " Sep 10 14:59:26.620688 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.620662 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf06af0e-43a1-4d68-83dc-b24eff544b3c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf06af0e-43a1-4d68-83dc-b24eff544b3c" (UID: "cf06af0e-43a1-4d68-83dc-b24eff544b3c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:59:26.620688 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.620677 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ca3c631-fc6a-46b2-9f40-5910fe677a7e" (UID: "9ca3c631-fc6a-46b2-9f40-5910fe677a7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:59:26.622388 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.622364 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-kube-api-access-rpdz2" (OuterVolumeSpecName: "kube-api-access-rpdz2") pod "9ca3c631-fc6a-46b2-9f40-5910fe677a7e" (UID: "9ca3c631-fc6a-46b2-9f40-5910fe677a7e"). InnerVolumeSpecName "kube-api-access-rpdz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:59:26.622479 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.622395 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf06af0e-43a1-4d68-83dc-b24eff544b3c-kube-api-access-jsvfh" (OuterVolumeSpecName: "kube-api-access-jsvfh") pod "cf06af0e-43a1-4d68-83dc-b24eff544b3c" (UID: "cf06af0e-43a1-4d68-83dc-b24eff544b3c"). InnerVolumeSpecName "kube-api-access-jsvfh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:59:26.722225 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.722168 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpdz2\" (UniqueName: \"kubernetes.io/projected/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-kube-api-access-rpdz2\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:26.722225 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.722222 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf06af0e-43a1-4d68-83dc-b24eff544b3c-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:26.722225 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.722232 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsvfh\" (UniqueName: \"kubernetes.io/projected/cf06af0e-43a1-4d68-83dc-b24eff544b3c-kube-api-access-jsvfh\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:26.722488 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.722241 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca3c631-fc6a-46b2-9f40-5910fe677a7e-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:26.916838 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.916807 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff"] Sep 10 14:59:26.920455 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.920427 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ff661a-744847d799-5s9ff"] Sep 10 14:59:26.934957 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.934916 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h"] Sep 10 14:59:26.948917 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:26.948880 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ff661a-78bd8c957d-hmt6h"] Sep 10 14:59:27.541784 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:27.541752 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 14:59:27.542155 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:27.541937 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 14:59:27.762984 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:27.762948 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca3c631-fc6a-46b2-9f40-5910fe677a7e" path="/var/lib/kubelet/pods/9ca3c631-fc6a-46b2-9f40-5910fe677a7e/volumes" Sep 10 14:59:27.763327 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:27.763313 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf06af0e-43a1-4d68-83dc-b24eff544b3c" path="/var/lib/kubelet/pods/cf06af0e-43a1-4d68-83dc-b24eff544b3c/volumes" Sep 10 14:59:33.078817 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:33.078784 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2"] Sep 10 14:59:33.079364 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:33.078973 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" containerID="cri-o://043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8" gracePeriod=30 Sep 10 14:59:38.339581 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.339556 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:59:38.359800 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.359738 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95"] Sep 10 14:59:38.360075 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360063 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360076 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360088 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360094 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360104 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360109 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360116 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca3c631-fc6a-46b2-9f40-5910fe677a7e" containerName="payload-processing" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360121 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca3c631-fc6a-46b2-9f40-5910fe677a7e" containerName="payload-processing" Sep 10 14:59:38.360128 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360129 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" Sep 10 14:59:38.360462 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360134 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" Sep 10 14:59:38.360462 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360148 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf06af0e-43a1-4d68-83dc-b24eff544b3c" containerName="payload-pre-processing" Sep 10 14:59:38.360462 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360153 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf06af0e-43a1-4d68-83dc-b24eff544b3c" containerName="payload-pre-processing" Sep 10 14:59:38.360462 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360216 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca3c631-fc6a-46b2-9f40-5910fe677a7e" containerName="payload-processing" Sep 10 14:59:38.360462 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360232 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf06af0e-43a1-4d68-83dc-b24eff544b3c" containerName="payload-pre-processing" Sep 10 14:59:38.360462 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360239 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" Sep 10 14:59:38.360462 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.360247 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" Sep 10 14:59:38.363024 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.363007 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.366155 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.366051 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-16ad45\"" Sep 10 14:59:38.366441 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.366413 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-16ad45-dockercfg-nsmfh\"" Sep 10 14:59:38.374730 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.374709 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95"] Sep 10 14:59:38.392931 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.392907 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6"] Sep 10 14:59:38.393312 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.393298 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerName="istio-proxy" Sep 10 14:59:38.393378 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.393313 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb8945bb-c09d-4482-9792-3df34af87b4c" containerName="istio-proxy" Sep 10 14:59:38.396171 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.396154 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.407017 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.406993 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6"] Sep 10 14:59:38.431506 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431473 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-credential-socket\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431523 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-socket\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431567 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-envoy\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431613 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-certs\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431653 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istiod-ca-cert\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431680 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431679 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-token\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431950 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431744 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-podinfo\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431950 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431770 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-data\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431950 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431779 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:38.431950 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431850 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnm9v\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-kube-api-access-jnm9v\") pod \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\" (UID: \"7c6cfd0a-9a97-4182-874b-7ccb1e139db0\") " Sep 10 14:59:38.431950 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431877 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:38.432227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.431969 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:38.432227 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.432135 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:38.432348 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.432232 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.432348 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.432250 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-data\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.432348 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.432265 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-credential-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.432348 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.432279 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-workload-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.432548 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.432521 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 14:59:38.434937 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.434894 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 14:59:38.435190 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.435141 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 14:59:38.435190 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.435140 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-kube-api-access-jnm9v" (OuterVolumeSpecName: "kube-api-access-jnm9v") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "kube-api-access-jnm9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:59:38.436184 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.435744 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c6cfd0a-9a97-4182-874b-7ccb1e139db0" (UID: "7c6cfd0a-9a97-4182-874b-7ccb1e139db0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 14:59:38.533041 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533009 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478f8ae7-0112-4457-9b50-22e311734e36-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.533176 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533049 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj77w\" (UniqueName: \"kubernetes.io/projected/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-kube-api-access-cj77w\") pod \"payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.533176 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533069 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nww7b\" (UniqueName: \"kubernetes.io/projected/478f8ae7-0112-4457-9b50-22e311734e36-kube-api-access-nww7b\") pod \"payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.533270 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533171 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.533344 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533319 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnm9v\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-kube-api-access-jnm9v\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.533344 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533342 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-envoy\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.533487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533352 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istiod-ca-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.533487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533360 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-token\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.533487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.533368 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c6cfd0a-9a97-4182-874b-7ccb1e139db0-istio-podinfo\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 14:59:38.634087 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.634023 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.634212 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.634093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478f8ae7-0112-4457-9b50-22e311734e36-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.634212 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.634124 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj77w\" (UniqueName: \"kubernetes.io/projected/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-kube-api-access-cj77w\") pod \"payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.634212 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.634143 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nww7b\" (UniqueName: \"kubernetes.io/projected/478f8ae7-0112-4457-9b50-22e311734e36-kube-api-access-nww7b\") pod \"payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.634794 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.634775 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.634906 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.634888 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478f8ae7-0112-4457-9b50-22e311734e36-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.642149 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.642126 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nww7b\" (UniqueName: \"kubernetes.io/projected/478f8ae7-0112-4457-9b50-22e311734e36-kube-api-access-nww7b\") pod \"payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.642315 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.642295 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj77w\" (UniqueName: \"kubernetes.io/projected/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-kube-api-access-cj77w\") pod \"payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.659776 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.659752 2584 generic.go:356] "Generic (PLEG): container finished" podID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" containerID="043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8" exitCode=0 Sep 10 14:59:38.659878 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.659801 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" Sep 10 14:59:38.659878 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.659829 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" event={"ID":"7c6cfd0a-9a97-4182-874b-7ccb1e139db0","Type":"ContainerDied","Data":"043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8"} Sep 10 14:59:38.659975 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.659890 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2" event={"ID":"7c6cfd0a-9a97-4182-874b-7ccb1e139db0","Type":"ContainerDied","Data":"1cb69a297ac6bf97a0002c444e507574dbc1b4b7daaaf3e37394714ce627e2de"} Sep 10 14:59:38.659975 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.659921 2584 scope.go:117] "RemoveContainer" containerID="043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8" Sep 10 14:59:38.669026 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.669009 2584 scope.go:117] "RemoveContainer" containerID="85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1" Sep 10 14:59:38.675189 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.675171 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:38.676819 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.676804 2584 scope.go:117] "RemoveContainer" containerID="043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8" Sep 10 14:59:38.677063 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:59:38.677047 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8\": container with ID starting with 043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8 not found: ID does not exist" containerID="043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8" Sep 10 14:59:38.677105 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.677070 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8"} err="failed to get container status \"043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8\": rpc error: code = NotFound desc = could not find container \"043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8\": container with ID starting with 043ef59799479af74e7bad55c6ad0d9e30de90cd6005a851c908d6477c6ccdd8 not found: ID does not exist" Sep 10 14:59:38.677105 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.677085 2584 scope.go:117] "RemoveContainer" containerID="85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1" Sep 10 14:59:38.677334 ip-10-0-136-202 kubenswrapper[2584]: E0910 14:59:38.677315 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1\": container with ID starting with 85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1 not found: ID does not exist" containerID="85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1" Sep 10 14:59:38.677400 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.677345 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1"} err="failed to get container status \"85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1\": rpc error: code = NotFound desc = could not find container \"85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1\": container with ID starting with 85b993fbc6d21f9d0ff79ec51ad461b67e6915a5fe7aca2e0293cead4d5911e1 not found: ID does not exist" Sep 10 14:59:38.683000 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.682980 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2"] Sep 10 14:59:38.686926 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.686910 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ff661a-openshift-default-66fc9c6756-xbnp2"] Sep 10 14:59:38.705936 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.705916 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:38.946401 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.946374 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95"] Sep 10 14:59:38.948655 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:59:38.948631 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1fbb42_ad63_4be4_a87f_70dd056c02fd.slice/crio-ea49e0edd9f08708a12614c2a9c318e5d8abd9513671324f510488dba4c7f8c2 WatchSource:0}: Error finding container ea49e0edd9f08708a12614c2a9c318e5d8abd9513671324f510488dba4c7f8c2: Status 404 returned error can't find the container with id ea49e0edd9f08708a12614c2a9c318e5d8abd9513671324f510488dba4c7f8c2 Sep 10 14:59:38.986637 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:38.986608 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6"] Sep 10 14:59:39.667416 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.667384 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" event={"ID":"dd1fbb42-ad63-4be4-a87f-70dd056c02fd","Type":"ContainerStarted","Data":"e37ee08947476de2a059dd72a49840d0c92bf70d807b460214c67c082546d283"} Sep 10 14:59:39.667861 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.667434 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" event={"ID":"dd1fbb42-ad63-4be4-a87f-70dd056c02fd","Type":"ContainerStarted","Data":"ea49e0edd9f08708a12614c2a9c318e5d8abd9513671324f510488dba4c7f8c2"} Sep 10 14:59:39.667861 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.667486 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:39.670031 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.669997 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" event={"ID":"478f8ae7-0112-4457-9b50-22e311734e36","Type":"ContainerStarted","Data":"452f6928acd975782d417ff45ef9dd7d1be57322a0a02591fef0818ae6f068d1"} Sep 10 14:59:39.670176 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.670040 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" event={"ID":"478f8ae7-0112-4457-9b50-22e311734e36","Type":"ContainerStarted","Data":"b4abb2566da1004d2d54941ce657f0ba2c05d9741e50da31f320a6af98c3f97d"} Sep 10 14:59:39.670176 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.670141 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:39.684537 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.684497 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" podStartSLOduration=1.684485593 podStartE2EDuration="1.684485593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:39.683663939 +0000 UTC m=+1036.355689712" watchObservedRunningTime="2026-09-10 14:59:39.684485593 +0000 UTC m=+1036.356511395" Sep 10 14:59:39.700416 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.700379 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" podStartSLOduration=1.700367083 podStartE2EDuration="1.700367083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:39.698112217 +0000 UTC m=+1036.370137972" watchObservedRunningTime="2026-09-10 14:59:39.700367083 +0000 UTC m=+1036.372392860" Sep 10 14:59:39.762885 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:39.762862 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c6cfd0a-9a97-4182-874b-7ccb1e139db0" path="/var/lib/kubelet/pods/7c6cfd0a-9a97-4182-874b-7ccb1e139db0/volumes" Sep 10 14:59:50.410333 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.410302 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9"] Sep 10 14:59:50.415183 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.415164 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.423260 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.423242 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6621560b-openshift-default-dockercfg-45v66\"" Sep 10 14:59:50.428168 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428148 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j6qs\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-kube-api-access-7j6qs\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428281 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-certs\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428281 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428220 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-socket\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428392 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428276 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dc1e008-ed20-4651-a66e-251c987dd180-istiod-ca-cert\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428392 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428313 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dc1e008-ed20-4651-a66e-251c987dd180-istio-podinfo\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428392 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428350 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-envoy\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428392 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428368 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-data\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428392 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428384 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-credential-socket\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.428660 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.428428 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-istio-token\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.433106 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.433085 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9"] Sep 10 14:59:50.529149 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529111 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j6qs\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-kube-api-access-7j6qs\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529335 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529171 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-certs\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529335 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529233 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-socket\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529335 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529261 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dc1e008-ed20-4651-a66e-251c987dd180-istiod-ca-cert\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529335 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529289 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dc1e008-ed20-4651-a66e-251c987dd180-istio-podinfo\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529552 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529342 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-envoy\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529552 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529375 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-data\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529552 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529401 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-credential-socket\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529552 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529452 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-istio-token\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529813 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-data\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529871 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529857 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-credential-socket\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529937 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529922 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-socket\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.529972 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529919 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-certs\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.530008 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.529990 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dc1e008-ed20-4651-a66e-251c987dd180-istiod-ca-cert\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.531722 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.531706 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-envoy\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.532176 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.532154 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dc1e008-ed20-4651-a66e-251c987dd180-istio-podinfo\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.545274 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.545247 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-istio-token\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.545487 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.545445 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j6qs\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-kube-api-access-7j6qs\") pod \"e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:50.677568 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.677502 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 14:59:50.677679 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.677571 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 14:59:50.725984 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:50.725957 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:51.014683 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:51.014655 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9"] Sep 10 14:59:51.016476 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:59:51.016453 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc1e008_ed20_4651_a66e_251c987dd180.slice/crio-a85e9d917e80a0ed02d77f13452e7cfe97e74899588b0786f5afcb699f405500 WatchSource:0}: Error finding container a85e9d917e80a0ed02d77f13452e7cfe97e74899588b0786f5afcb699f405500: Status 404 returned error can't find the container with id a85e9d917e80a0ed02d77f13452e7cfe97e74899588b0786f5afcb699f405500 Sep 10 14:59:51.018703 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:51.018673 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:59:51.018779 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:51.018737 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:59:51.018779 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:51.018762 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 14:59:51.735857 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:51.735821 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" event={"ID":"1dc1e008-ed20-4651-a66e-251c987dd180","Type":"ContainerStarted","Data":"0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686"} Sep 10 14:59:51.736240 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:51.735894 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" event={"ID":"1dc1e008-ed20-4651-a66e-251c987dd180","Type":"ContainerStarted","Data":"a85e9d917e80a0ed02d77f13452e7cfe97e74899588b0786f5afcb699f405500"} Sep 10 14:59:51.765649 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:51.765605 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podStartSLOduration=1.765591242 podStartE2EDuration="1.765591242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:51.763455027 +0000 UTC m=+1048.435480804" watchObservedRunningTime="2026-09-10 14:59:51.765591242 +0000 UTC m=+1048.437617013" Sep 10 14:59:52.726429 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:52.726392 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:52.731169 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:52.731144 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:52.744142 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:52.744115 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:52.744811 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:52.744794 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 14:59:55.567368 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.567332 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms"] Sep 10 14:59:55.571062 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.571040 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:55.573840 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.573821 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6621560b\"" Sep 10 14:59:55.584438 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.584417 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms"] Sep 10 14:59:55.675316 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.675289 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-plugins-config-volume\") pod \"payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:55.675440 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.675416 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzvp8\" (UniqueName: \"kubernetes.io/projected/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-kube-api-access-hzvp8\") pod \"payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:55.776356 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.776321 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzvp8\" (UniqueName: \"kubernetes.io/projected/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-kube-api-access-hzvp8\") pod \"payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:55.776522 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.776438 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-plugins-config-volume\") pod \"payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:55.777062 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.777041 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-plugins-config-volume\") pod \"payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:55.786123 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.786105 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzvp8\" (UniqueName: \"kubernetes.io/projected/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-kube-api-access-hzvp8\") pod \"payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:55.881355 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:55.881299 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:56.152545 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:56.152519 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms"] Sep 10 14:59:56.154498 ip-10-0-136-202 kubenswrapper[2584]: W0910 14:59:56.154472 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a29543a_6e9e_4b8c_b650_3a0158b6dda3.slice/crio-250b7457daaf059fd92fca162e5ced383fbb9e52d680c48b19faa7bd39121663 WatchSource:0}: Error finding container 250b7457daaf059fd92fca162e5ced383fbb9e52d680c48b19faa7bd39121663: Status 404 returned error can't find the container with id 250b7457daaf059fd92fca162e5ced383fbb9e52d680c48b19faa7bd39121663 Sep 10 14:59:56.763097 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:56.763066 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" event={"ID":"4a29543a-6e9e-4b8c-b650-3a0158b6dda3","Type":"ContainerStarted","Data":"40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d"} Sep 10 14:59:56.763463 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:56.763112 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" event={"ID":"4a29543a-6e9e-4b8c-b650-3a0158b6dda3","Type":"ContainerStarted","Data":"250b7457daaf059fd92fca162e5ced383fbb9e52d680c48b19faa7bd39121663"} Sep 10 14:59:56.763463 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:56.763215 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 14:59:56.782844 ip-10-0-136-202 kubenswrapper[2584]: I0910 14:59:56.782796 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" podStartSLOduration=1.782784962 podStartE2EDuration="1.782784962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:59:56.780596986 +0000 UTC m=+1053.452622759" watchObservedRunningTime="2026-09-10 14:59:56.782784962 +0000 UTC m=+1053.454810776" Sep 10 15:00:00.151128 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.151089 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p"] Sep 10 15:00:00.154851 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.154834 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" Sep 10 15:00:00.159330 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.159303 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\"" Sep 10 15:00:00.165220 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.165177 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh"] Sep 10 15:00:00.165759 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.165744 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" Sep 10 15:00:00.169050 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.169029 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" Sep 10 15:00:00.170175 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.170153 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p"] Sep 10 15:00:00.180266 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.180233 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh"] Sep 10 15:00:00.203284 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.203256 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" Sep 10 15:00:00.446367 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.446309 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p"] Sep 10 15:00:00.448559 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:00:00.448534 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5128c045_4d08_4090_afde_7ee26439cf75.slice/crio-ea5f381ab3562418fa44f19813c5dfccee3b3a9402e2f8c99166d82157ca20dd WatchSource:0}: Error finding container ea5f381ab3562418fa44f19813c5dfccee3b3a9402e2f8c99166d82157ca20dd: Status 404 returned error can't find the container with id ea5f381ab3562418fa44f19813c5dfccee3b3a9402e2f8c99166d82157ca20dd Sep 10 15:00:00.450831 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.450801 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 15:00:00.497819 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.497799 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh"] Sep 10 15:00:00.499261 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:00:00.499233 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45eda6db_975b_41a5_baae_fd09bb4fa5e5.slice/crio-f1bab0e60f659c0df115d2785418a2072d5e2c7224d438b6c907f20bf30ae909 WatchSource:0}: Error finding container f1bab0e60f659c0df115d2785418a2072d5e2c7224d438b6c907f20bf30ae909: Status 404 returned error can't find the container with id f1bab0e60f659c0df115d2785418a2072d5e2c7224d438b6c907f20bf30ae909 Sep 10 15:00:00.782565 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.782463 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" event={"ID":"45eda6db-975b-41a5-baae-fd09bb4fa5e5","Type":"ContainerStarted","Data":"f1bab0e60f659c0df115d2785418a2072d5e2c7224d438b6c907f20bf30ae909"} Sep 10 15:00:00.783628 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:00.783601 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" event={"ID":"5128c045-4d08-4090-afde-7ee26439cf75","Type":"ContainerStarted","Data":"ea5f381ab3562418fa44f19813c5dfccee3b3a9402e2f8c99166d82157ca20dd"} Sep 10 15:00:04.810041 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:04.810000 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" event={"ID":"5128c045-4d08-4090-afde-7ee26439cf75","Type":"ContainerStarted","Data":"2f8bda4ce86dab0719b09b947eabdc80321b86aeefb1a9ced0c1fc754bcffaa3"} Sep 10 15:00:04.811604 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:04.811578 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" event={"ID":"45eda6db-975b-41a5-baae-fd09bb4fa5e5","Type":"ContainerStarted","Data":"8b8b90b36892aec79b3a49bf76bb6a9b0d340ccb72fa205a06127c945090bfa1"} Sep 10 15:00:04.830015 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:04.829915 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" podStartSLOduration=1.61993054 podStartE2EDuration="4.82989759s" totalImagesPullingTime="3.20996705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:04.828066218 +0000 UTC m=+1061.500092004" watchObservedRunningTime="2026-09-10 15:00:04.82989759 +0000 UTC m=+1061.501923361" Sep 10 15:00:04.855050 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:04.854995 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" podStartSLOduration=1.687579018 podStartE2EDuration="4.854975997s" totalImagesPullingTime="3.167396979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:04.851967097 +0000 UTC m=+1061.523992874" watchObservedRunningTime="2026-09-10 15:00:04.854975997 +0000 UTC m=+1061.527001769" Sep 10 15:00:05.818630 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:05.818469 2584 generic.go:356] "Generic (PLEG): container finished" podID="5128c045-4d08-4090-afde-7ee26439cf75" containerID="2f8bda4ce86dab0719b09b947eabdc80321b86aeefb1a9ced0c1fc754bcffaa3" exitCode=0 Sep 10 15:00:05.818630 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:05.818548 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" event={"ID":"5128c045-4d08-4090-afde-7ee26439cf75","Type":"ContainerDied","Data":"2f8bda4ce86dab0719b09b947eabdc80321b86aeefb1a9ced0c1fc754bcffaa3"} Sep 10 15:00:05.820076 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:05.820050 2584 generic.go:356] "Generic (PLEG): container finished" podID="45eda6db-975b-41a5-baae-fd09bb4fa5e5" containerID="8b8b90b36892aec79b3a49bf76bb6a9b0d340ccb72fa205a06127c945090bfa1" exitCode=0 Sep 10 15:00:05.820215 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:05.820116 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" event={"ID":"45eda6db-975b-41a5-baae-fd09bb4fa5e5","Type":"ContainerDied","Data":"8b8b90b36892aec79b3a49bf76bb6a9b0d340ccb72fa205a06127c945090bfa1"} Sep 10 15:00:06.962307 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:06.962268 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" Sep 10 15:00:06.998001 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:06.997977 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" Sep 10 15:00:07.769120 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:07.769085 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 15:00:07.833704 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:07.833670 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" Sep 10 15:00:07.833898 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:07.833728 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817540-rmdhh" event={"ID":"45eda6db-975b-41a5-baae-fd09bb4fa5e5","Type":"ContainerDied","Data":"f1bab0e60f659c0df115d2785418a2072d5e2c7224d438b6c907f20bf30ae909"} Sep 10 15:00:07.833898 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:07.833813 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1bab0e60f659c0df115d2785418a2072d5e2c7224d438b6c907f20bf30ae909" Sep 10 15:00:07.836482 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:07.836168 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" event={"ID":"5128c045-4d08-4090-afde-7ee26439cf75","Type":"ContainerDied","Data":"ea5f381ab3562418fa44f19813c5dfccee3b3a9402e2f8c99166d82157ca20dd"} Sep 10 15:00:07.836482 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:07.836253 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea5f381ab3562418fa44f19813c5dfccee3b3a9402e2f8c99166d82157ca20dd" Sep 10 15:00:07.836482 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:07.836324 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p" Sep 10 15:00:08.078897 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.078808 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h"] Sep 10 15:00:08.079276 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.079188 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45eda6db-975b-41a5-baae-fd09bb4fa5e5" containerName="cleanup" Sep 10 15:00:08.079276 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.079212 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="45eda6db-975b-41a5-baae-fd09bb4fa5e5" containerName="cleanup" Sep 10 15:00:08.079276 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.079235 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5128c045-4d08-4090-afde-7ee26439cf75" containerName="cleanup" Sep 10 15:00:08.079276 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.079241 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5128c045-4d08-4090-afde-7ee26439cf75" containerName="cleanup" Sep 10 15:00:08.079413 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.079294 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5128c045-4d08-4090-afde-7ee26439cf75" containerName="cleanup" Sep 10 15:00:08.079413 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.079301 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="45eda6db-975b-41a5-baae-fd09bb4fa5e5" containerName="cleanup" Sep 10 15:00:08.082330 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.082313 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.085895 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.085868 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-3f81b7-dockercfg-47zdw\"" Sep 10 15:00:08.086217 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.086181 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-3f81b7\"" Sep 10 15:00:08.098347 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.098318 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h"] Sep 10 15:00:08.170992 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.170955 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg"] Sep 10 15:00:08.175422 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.175395 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.193095 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.193059 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg"] Sep 10 15:00:08.198766 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.198718 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.199035 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.199014 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8wf4\" (UniqueName: \"kubernetes.io/projected/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-kube-api-access-d8wf4\") pod \"payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.300257 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.300214 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.300455 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.300271 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6msss\" (UniqueName: \"kubernetes.io/projected/b1107f52-46b9-40cb-9865-1f5aa38e6717-kube-api-access-6msss\") pod \"payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.300455 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.300311 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8wf4\" (UniqueName: \"kubernetes.io/projected/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-kube-api-access-d8wf4\") pod \"payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.300455 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.300426 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1107f52-46b9-40cb-9865-1f5aa38e6717-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.301042 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.301024 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.309693 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.309668 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8wf4\" (UniqueName: \"kubernetes.io/projected/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-kube-api-access-d8wf4\") pod \"payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.392518 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.392411 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.401620 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.401588 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6msss\" (UniqueName: \"kubernetes.io/projected/b1107f52-46b9-40cb-9865-1f5aa38e6717-kube-api-access-6msss\") pod \"payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.401804 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.401693 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1107f52-46b9-40cb-9865-1f5aa38e6717-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.402466 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.402439 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1107f52-46b9-40cb-9865-1f5aa38e6717-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.411329 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.411302 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6msss\" (UniqueName: \"kubernetes.io/projected/b1107f52-46b9-40cb-9865-1f5aa38e6717-kube-api-access-6msss\") pod \"payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.496761 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.496721 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:08.671592 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.671562 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h"] Sep 10 15:00:08.673405 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:00:08.673378 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58aec8d_ec8e_4238_9e8f_d35a722b3b39.slice/crio-6fad2facb5168cf6dd74f38e3d5ac85d34c6fdfa6f0518f675322a40d10b784e WatchSource:0}: Error finding container 6fad2facb5168cf6dd74f38e3d5ac85d34c6fdfa6f0518f675322a40d10b784e: Status 404 returned error can't find the container with id 6fad2facb5168cf6dd74f38e3d5ac85d34c6fdfa6f0518f675322a40d10b784e Sep 10 15:00:08.783302 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.783222 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg"] Sep 10 15:00:08.786226 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:00:08.786187 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1107f52_46b9_40cb_9865_1f5aa38e6717.slice/crio-5b964dd6ff30d8f044c27f7430f573dea3e99da23084b888067fb8b7c5553f3e WatchSource:0}: Error finding container 5b964dd6ff30d8f044c27f7430f573dea3e99da23084b888067fb8b7c5553f3e: Status 404 returned error can't find the container with id 5b964dd6ff30d8f044c27f7430f573dea3e99da23084b888067fb8b7c5553f3e Sep 10 15:00:08.786353 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.786326 2584 patch_prober.go:28] interesting pod/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 10 15:00:08.786414 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.786377 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 10 15:00:08.843024 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.842987 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" event={"ID":"b58aec8d-ec8e-4238-9e8f-d35a722b3b39","Type":"ContainerStarted","Data":"4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a"} Sep 10 15:00:08.843234 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.843049 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" event={"ID":"b58aec8d-ec8e-4238-9e8f-d35a722b3b39","Type":"ContainerStarted","Data":"6fad2facb5168cf6dd74f38e3d5ac85d34c6fdfa6f0518f675322a40d10b784e"} Sep 10 15:00:08.843234 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.843166 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:08.844494 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.844462 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" event={"ID":"b1107f52-46b9-40cb-9865-1f5aa38e6717","Type":"ContainerStarted","Data":"5b964dd6ff30d8f044c27f7430f573dea3e99da23084b888067fb8b7c5553f3e"} Sep 10 15:00:08.865666 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:08.865602 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" podStartSLOduration=0.865578628 podStartE2EDuration="865.578628ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:08.865058007 +0000 UTC m=+1065.537083789" watchObservedRunningTime="2026-09-10 15:00:08.865578628 +0000 UTC m=+1065.537604403" Sep 10 15:00:09.852514 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.852419 2584 generic.go:356] "Generic (PLEG): container finished" podID="1dc1e008-ed20-4651-a66e-251c987dd180" containerID="0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686" exitCode=0 Sep 10 15:00:09.852514 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.852500 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" event={"ID":"1dc1e008-ed20-4651-a66e-251c987dd180","Type":"ContainerDied","Data":"0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686"} Sep 10 15:00:09.853040 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.853017 2584 scope.go:117] "RemoveContainer" containerID="0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686" Sep 10 15:00:09.854450 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.854427 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" event={"ID":"b1107f52-46b9-40cb-9865-1f5aa38e6717","Type":"ContainerStarted","Data":"8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd"} Sep 10 15:00:09.854561 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.854456 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 15:00:09.854611 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.854595 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:00:09.854655 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.854602 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 15:00:09.854700 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.854651 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 10 15:00:09.906484 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:09.906392 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" podStartSLOduration=1.906368187 podStartE2EDuration="1.906368187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:09.9041821 +0000 UTC m=+1066.576207876" watchObservedRunningTime="2026-09-10 15:00:09.906368187 +0000 UTC m=+1066.578393960" Sep 10 15:00:10.864922 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:10.864885 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" event={"ID":"1dc1e008-ed20-4651-a66e-251c987dd180","Type":"ContainerStarted","Data":"aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032"} Sep 10 15:00:11.726701 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:11.726656 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 15:00:11.726948 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:11.726923 2584 patch_prober.go:28] interesting pod/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 15:00:11.727000 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:11.726977 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 15:00:12.726844 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:12.726802 2584 patch_prober.go:28] interesting pod/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 15:00:12.727332 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:12.726878 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 15:00:13.726822 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:13.726781 2584 patch_prober.go:28] interesting pod/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 15:00:13.727005 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:13.726839 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 15:00:14.726627 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:14.726584 2584 patch_prober.go:28] interesting pod/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 15:00:14.726820 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:14.726658 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 15:00:15.727038 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:15.726992 2584 patch_prober.go:28] interesting pod/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Sep 10 15:00:15.727555 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:15.727072 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Sep 10 15:00:16.885829 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:16.885739 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 15:00:16.886314 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:16.886237 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 15:00:16.887019 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:16.886995 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 15:00:19.856955 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:19.856927 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:00:20.866924 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:00:20.866890 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:01:24.268373 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.268336 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms"] Sep 10 15:01:24.269222 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.269145 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" podUID="4a29543a-6e9e-4b8c-b650-3a0158b6dda3" containerName="payload-processing" containerID="cri-o://40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d" gracePeriod=30 Sep 10 15:01:24.508152 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.508112 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 15:01:24.584119 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.584035 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzvp8\" (UniqueName: \"kubernetes.io/projected/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-kube-api-access-hzvp8\") pod \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " Sep 10 15:01:24.584286 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.584214 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-plugins-config-volume\") pod \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\" (UID: \"4a29543a-6e9e-4b8c-b650-3a0158b6dda3\") " Sep 10 15:01:24.584577 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.584550 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a29543a-6e9e-4b8c-b650-3a0158b6dda3" (UID: "4a29543a-6e9e-4b8c-b650-3a0158b6dda3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:01:24.586213 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.586174 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-kube-api-access-hzvp8" (OuterVolumeSpecName: "kube-api-access-hzvp8") pod "4a29543a-6e9e-4b8c-b650-3a0158b6dda3" (UID: "4a29543a-6e9e-4b8c-b650-3a0158b6dda3"). InnerVolumeSpecName "kube-api-access-hzvp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:01:24.685547 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.685511 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:01:24.685547 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:24.685540 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzvp8\" (UniqueName: \"kubernetes.io/projected/4a29543a-6e9e-4b8c-b650-3a0158b6dda3-kube-api-access-hzvp8\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:01:25.227043 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.227007 2584 generic.go:356] "Generic (PLEG): container finished" podID="4a29543a-6e9e-4b8c-b650-3a0158b6dda3" containerID="40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d" exitCode=0 Sep 10 15:01:25.227240 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.227055 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" Sep 10 15:01:25.227240 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.227088 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" event={"ID":"4a29543a-6e9e-4b8c-b650-3a0158b6dda3","Type":"ContainerDied","Data":"40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d"} Sep 10 15:01:25.227240 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.227139 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms" event={"ID":"4a29543a-6e9e-4b8c-b650-3a0158b6dda3","Type":"ContainerDied","Data":"250b7457daaf059fd92fca162e5ced383fbb9e52d680c48b19faa7bd39121663"} Sep 10 15:01:25.227240 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.227161 2584 scope.go:117] "RemoveContainer" containerID="40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d" Sep 10 15:01:25.236132 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.236109 2584 scope.go:117] "RemoveContainer" containerID="40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d" Sep 10 15:01:25.236434 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:01:25.236417 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d\": container with ID starting with 40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d not found: ID does not exist" containerID="40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d" Sep 10 15:01:25.236501 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.236441 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d"} err="failed to get container status \"40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d\": rpc error: code = NotFound desc = could not find container \"40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d\": container with ID starting with 40fa4b8aae879587f69d9d76aa174a35736841a45c324a93c80e88865319303d not found: ID does not exist" Sep 10 15:01:25.250407 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.250381 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms"] Sep 10 15:01:25.255275 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.255252 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6621560b-66d77649cc-9f9ms"] Sep 10 15:01:25.763277 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:01:25.763246 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a29543a-6e9e-4b8c-b650-3a0158b6dda3" path="/var/lib/kubelet/pods/4a29543a-6e9e-4b8c-b650-3a0158b6dda3/volumes" Sep 10 15:02:16.457244 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:16.457194 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9"] Sep 10 15:02:16.457806 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:16.457444 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" containerID="cri-o://aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032" gracePeriod=30 Sep 10 15:02:16.888974 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:16.888891 2584 patch_prober.go:28] interesting pod/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 10 15:02:16.889111 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:16.888973 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 10 15:02:21.717627 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.717559 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 15:02:21.812650 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812614 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-certs\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.812843 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812680 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j6qs\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-kube-api-access-7j6qs\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.812843 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812783 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-credential-socket\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.812843 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812821 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-data\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.813006 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812853 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-envoy\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.813006 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812889 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-socket\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.813006 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812927 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dc1e008-ed20-4651-a66e-251c987dd180-istiod-ca-cert\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.813006 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812936 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 15:02:21.813006 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812961 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dc1e008-ed20-4651-a66e-251c987dd180-istio-podinfo\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.813006 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.812985 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-istio-token\") pod \"1dc1e008-ed20-4651-a66e-251c987dd180\" (UID: \"1dc1e008-ed20-4651-a66e-251c987dd180\") " Sep 10 15:02:21.813332 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813129 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 15:02:21.813332 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813151 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-data" (OuterVolumeSpecName: "istio-data") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 15:02:21.813332 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813244 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 15:02:21.813492 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813401 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.813492 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813421 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-credential-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.813492 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813417 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dc1e008-ed20-4651-a66e-251c987dd180-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:02:21.813492 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813435 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-data\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.813492 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.813463 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-workload-socket\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.815102 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.815074 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 10 15:02:21.815264 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.815149 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-kube-api-access-7j6qs" (OuterVolumeSpecName: "kube-api-access-7j6qs") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "kube-api-access-7j6qs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:02:21.815264 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.815165 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1dc1e008-ed20-4651-a66e-251c987dd180-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 10 15:02:21.815387 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.815338 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-istio-token" (OuterVolumeSpecName: "istio-token") pod "1dc1e008-ed20-4651-a66e-251c987dd180" (UID: "1dc1e008-ed20-4651-a66e-251c987dd180"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:02:21.914456 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.914419 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1dc1e008-ed20-4651-a66e-251c987dd180-istio-envoy\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.914456 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.914449 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1dc1e008-ed20-4651-a66e-251c987dd180-istiod-ca-cert\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.914456 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.914459 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1dc1e008-ed20-4651-a66e-251c987dd180-istio-podinfo\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.914456 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.914466 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-istio-token\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:21.914774 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:21.914477 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j6qs\" (UniqueName: \"kubernetes.io/projected/1dc1e008-ed20-4651-a66e-251c987dd180-kube-api-access-7j6qs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:02:22.511129 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.511089 2584 generic.go:356] "Generic (PLEG): container finished" podID="1dc1e008-ed20-4651-a66e-251c987dd180" containerID="aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032" exitCode=0 Sep 10 15:02:22.511343 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.511140 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" Sep 10 15:02:22.511343 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.511170 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" event={"ID":"1dc1e008-ed20-4651-a66e-251c987dd180","Type":"ContainerDied","Data":"aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032"} Sep 10 15:02:22.511343 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.511249 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9" event={"ID":"1dc1e008-ed20-4651-a66e-251c987dd180","Type":"ContainerDied","Data":"a85e9d917e80a0ed02d77f13452e7cfe97e74899588b0786f5afcb699f405500"} Sep 10 15:02:22.511343 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.511280 2584 scope.go:117] "RemoveContainer" containerID="aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032" Sep 10 15:02:22.521643 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.521615 2584 scope.go:117] "RemoveContainer" containerID="0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686" Sep 10 15:02:22.530859 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.530838 2584 scope.go:117] "RemoveContainer" containerID="aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032" Sep 10 15:02:22.531175 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:02:22.531153 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032\": container with ID starting with aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032 not found: ID does not exist" containerID="aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032" Sep 10 15:02:22.531317 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.531188 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032"} err="failed to get container status \"aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032\": rpc error: code = NotFound desc = could not find container \"aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032\": container with ID starting with aeaebc7c5ba7b038e10b9d7e1c032f0f5788f1005c65290d42a49c4a5e0ab032 not found: ID does not exist" Sep 10 15:02:22.531423 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.531319 2584 scope.go:117] "RemoveContainer" containerID="0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686" Sep 10 15:02:22.531655 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:02:22.531637 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686\": container with ID starting with 0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686 not found: ID does not exist" containerID="0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686" Sep 10 15:02:22.531710 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.531659 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686"} err="failed to get container status \"0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686\": rpc error: code = NotFound desc = could not find container \"0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686\": container with ID starting with 0b36e79411a1de966da7495c9cc837d646416c2b40d126748a5c390d70d13686 not found: ID does not exist" Sep 10 15:02:22.547756 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.547690 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9"] Sep 10 15:02:22.550169 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:22.550144 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6621560b-openshift-default-8867fc8d8-vrkd9"] Sep 10 15:02:23.764108 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:23.764073 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" path="/var/lib/kubelet/pods/1dc1e008-ed20-4651-a66e-251c987dd180/volumes" Sep 10 15:02:23.778917 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:23.778884 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:02:23.789840 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:02:23.789804 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:03:17.565611 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.565513 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7"] Sep 10 15:03:17.566159 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.566087 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" Sep 10 15:03:17.566159 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.566109 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" Sep 10 15:03:17.566159 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.566158 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a29543a-6e9e-4b8c-b650-3a0158b6dda3" containerName="payload-processing" Sep 10 15:03:17.566385 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.566167 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a29543a-6e9e-4b8c-b650-3a0158b6dda3" containerName="payload-processing" Sep 10 15:03:17.566385 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.566318 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" Sep 10 15:03:17.566385 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.566335 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a29543a-6e9e-4b8c-b650-3a0158b6dda3" containerName="payload-processing" Sep 10 15:03:17.566385 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.566350 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" Sep 10 15:03:17.581028 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.580995 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.584233 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.583990 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7"] Sep 10 15:03:17.585759 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.585734 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-550dc3-serving-cert\"" Sep 10 15:03:17.585930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.585749 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9t8qm\"" Sep 10 15:03:17.586248 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.585806 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 15:03:17.586552 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.585853 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 15:03:17.586769 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.585889 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:03:17.629641 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.629597 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.629641 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.629653 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca05b596-7802-44bc-865f-26ceddd34f8c-openshift-service-ca\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.629924 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.629669 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-metrics-certs\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.629924 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.629798 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqk55\" (UniqueName: \"kubernetes.io/projected/ca05b596-7802-44bc-865f-26ceddd34f8c-kube-api-access-sqk55\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.730315 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.730279 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca05b596-7802-44bc-865f-26ceddd34f8c-openshift-service-ca\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.730315 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.730318 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-metrics-certs\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.730607 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.730376 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqk55\" (UniqueName: \"kubernetes.io/projected/ca05b596-7802-44bc-865f-26ceddd34f8c-kube-api-access-sqk55\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.730607 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.730537 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.730722 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:03:17.730655 2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-serving-cert: secret "maas-api-e2e-api-a-550dc3-serving-cert" not found Sep 10 15:03:17.730781 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:03:17.730732 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls podName:ca05b596-7802-44bc-865f-26ceddd34f8c nodeName:}" failed. No retries permitted until 2026-09-10 15:03:18.230708205 +0000 UTC m=+1254.902733964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls") pod "maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" (UID: "ca05b596-7802-44bc-865f-26ceddd34f8c") : secret "maas-api-e2e-api-a-550dc3-serving-cert" not found Sep 10 15:03:17.731068 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.731042 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca05b596-7802-44bc-865f-26ceddd34f8c-openshift-service-ca\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.733071 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.733042 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-metrics-certs\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:17.743062 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:17.743025 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqk55\" (UniqueName: \"kubernetes.io/projected/ca05b596-7802-44bc-865f-26ceddd34f8c-kube-api-access-sqk55\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:18.235124 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.235079 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:18.238023 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.237980 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf"] Sep 10 15:03:18.238410 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.238367 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls\") pod \"maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:18.238598 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.238582 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" Sep 10 15:03:18.238640 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.238602 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc1e008-ed20-4651-a66e-251c987dd180" containerName="istio-proxy" Sep 10 15:03:18.242105 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.242087 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.244948 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.244924 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-550dc3\"" Sep 10 15:03:18.245032 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.245004 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-550dc3-dockercfg-6f5lb\"" Sep 10 15:03:18.252564 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.252536 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf"] Sep 10 15:03:18.272054 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.272016 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5"] Sep 10 15:03:18.275549 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.275529 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.288751 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.288712 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5"] Sep 10 15:03:18.335937 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.335903 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzq68\" (UniqueName: \"kubernetes.io/projected/b3e96518-e446-479a-ab46-571c6fb0d4ec-kube-api-access-wzq68\") pod \"payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.336100 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.335962 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e96518-e446-479a-ab46-571c6fb0d4ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.336100 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.336026 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8j8v\" (UniqueName: \"kubernetes.io/projected/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-kube-api-access-s8j8v\") pod \"payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.336100 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.336068 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.436972 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.436929 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzq68\" (UniqueName: \"kubernetes.io/projected/b3e96518-e446-479a-ab46-571c6fb0d4ec-kube-api-access-wzq68\") pod \"payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.437182 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.437020 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e96518-e446-479a-ab46-571c6fb0d4ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.437182 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.437059 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8j8v\" (UniqueName: \"kubernetes.io/projected/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-kube-api-access-s8j8v\") pod \"payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.437301 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.437229 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.437796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.437776 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e96518-e446-479a-ab46-571c6fb0d4ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.437867 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.437848 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.447021 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.446991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8j8v\" (UniqueName: \"kubernetes.io/projected/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-kube-api-access-s8j8v\") pod \"payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.447190 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.446999 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzq68\" (UniqueName: \"kubernetes.io/projected/b3e96518-e446-479a-ab46-571c6fb0d4ec-kube-api-access-wzq68\") pod \"payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.497449 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.497359 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:18.558606 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.558576 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:18.586923 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.586887 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:18.777086 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.777058 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7"] Sep 10 15:03:18.778998 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:03:18.778971 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca05b596_7802_44bc_865f_26ceddd34f8c.slice/crio-98fa6e8bed58698839299ace93c6626ee328c40ff94a39c41a44f467dfddd0cc WatchSource:0}: Error finding container 98fa6e8bed58698839299ace93c6626ee328c40ff94a39c41a44f467dfddd0cc: Status 404 returned error can't find the container with id 98fa6e8bed58698839299ace93c6626ee328c40ff94a39c41a44f467dfddd0cc Sep 10 15:03:18.791233 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.791177 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" event={"ID":"ca05b596-7802-44bc-865f-26ceddd34f8c","Type":"ContainerStarted","Data":"98fa6e8bed58698839299ace93c6626ee328c40ff94a39c41a44f467dfddd0cc"} Sep 10 15:03:18.838429 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.838401 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf"] Sep 10 15:03:18.839891 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:03:18.839859 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e96518_e446_479a_ab46_571c6fb0d4ec.slice/crio-27124e2c8474d584cf7ca4f230e2d4666bbb5c36d4ce28cc75660b4336353e92 WatchSource:0}: Error finding container 27124e2c8474d584cf7ca4f230e2d4666bbb5c36d4ce28cc75660b4336353e92: Status 404 returned error can't find the container with id 27124e2c8474d584cf7ca4f230e2d4666bbb5c36d4ce28cc75660b4336353e92 Sep 10 15:03:18.873763 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:18.873440 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5"] Sep 10 15:03:18.875657 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:03:18.875627 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1b3bb9f_a11c_435e_82d2_f5c27b0fadc6.slice/crio-5e43ff2865dfdc8bc0a0b2816712972f9e2797461996e7dd71eaab89bd819770 WatchSource:0}: Error finding container 5e43ff2865dfdc8bc0a0b2816712972f9e2797461996e7dd71eaab89bd819770: Status 404 returned error can't find the container with id 5e43ff2865dfdc8bc0a0b2816712972f9e2797461996e7dd71eaab89bd819770 Sep 10 15:03:19.801118 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.800760 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" event={"ID":"b3e96518-e446-479a-ab46-571c6fb0d4ec","Type":"ContainerStarted","Data":"3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e"} Sep 10 15:03:19.801118 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.800835 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" event={"ID":"b3e96518-e446-479a-ab46-571c6fb0d4ec","Type":"ContainerStarted","Data":"27124e2c8474d584cf7ca4f230e2d4666bbb5c36d4ce28cc75660b4336353e92"} Sep 10 15:03:19.801118 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.800889 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:19.803370 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.803333 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" event={"ID":"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6","Type":"ContainerStarted","Data":"57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c"} Sep 10 15:03:19.803529 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.803397 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" event={"ID":"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6","Type":"ContainerStarted","Data":"5e43ff2865dfdc8bc0a0b2816712972f9e2797461996e7dd71eaab89bd819770"} Sep 10 15:03:19.803529 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.803459 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:19.821357 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.821260 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" podStartSLOduration=1.8212351949999999 podStartE2EDuration="1.821235195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:03:19.81832135 +0000 UTC m=+1256.490347122" watchObservedRunningTime="2026-09-10 15:03:19.821235195 +0000 UTC m=+1256.493260969" Sep 10 15:03:19.839359 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:19.839277 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" podStartSLOduration=1.839253893 podStartE2EDuration="1.839253893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:03:19.837219832 +0000 UTC m=+1256.509245670" watchObservedRunningTime="2026-09-10 15:03:19.839253893 +0000 UTC m=+1256.511279667" Sep 10 15:03:21.823167 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:21.823130 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" event={"ID":"ca05b596-7802-44bc-865f-26ceddd34f8c","Type":"ContainerStarted","Data":"41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd"} Sep 10 15:03:21.823578 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:21.823238 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:21.842796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:21.842737 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" podStartSLOduration=2.431916395 podStartE2EDuration="4.84271714s" totalImagesPullingTime="2.410800745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:03:21.841931913 +0000 UTC m=+1258.513957690" watchObservedRunningTime="2026-09-10 15:03:21.84271714 +0000 UTC m=+1258.514742912" Sep 10 15:03:27.832586 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:27.832559 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:03:30.817582 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:30.817553 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:03:30.817974 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:30.817618 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:03:47.162173 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.162140 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh"] Sep 10 15:03:47.165548 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.165527 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.171382 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.171360 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-cb65dc-dockercfg-nb4kp\"" Sep 10 15:03:47.171471 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.171392 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-cb65dc\"" Sep 10 15:03:47.186428 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.186404 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh"] Sep 10 15:03:47.304112 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.304078 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d3a7d2-c726-40d8-b4ef-401446d587c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.304281 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.304130 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5s99\" (UniqueName: \"kubernetes.io/projected/f9d3a7d2-c726-40d8-b4ef-401446d587c2-kube-api-access-s5s99\") pod \"payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.405452 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.405422 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d3a7d2-c726-40d8-b4ef-401446d587c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.405592 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.405465 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5s99\" (UniqueName: \"kubernetes.io/projected/f9d3a7d2-c726-40d8-b4ef-401446d587c2-kube-api-access-s5s99\") pod \"payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.406082 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.406063 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d3a7d2-c726-40d8-b4ef-401446d587c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.414019 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.413952 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5s99\" (UniqueName: \"kubernetes.io/projected/f9d3a7d2-c726-40d8-b4ef-401446d587c2-kube-api-access-s5s99\") pod \"payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.476640 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.476606 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.758048 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:03:47.758012 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d3a7d2_c726_40d8_b4ef_401446d587c2.slice/crio-476f1f4509eaba2c56fc102a479116c4742587a733554885705f61c978f55133 WatchSource:0}: Error finding container 476f1f4509eaba2c56fc102a479116c4742587a733554885705f61c978f55133: Status 404 returned error can't find the container with id 476f1f4509eaba2c56fc102a479116c4742587a733554885705f61c978f55133 Sep 10 15:03:47.764687 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.764661 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh"] Sep 10 15:03:47.956669 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.956633 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" event={"ID":"f9d3a7d2-c726-40d8-b4ef-401446d587c2","Type":"ContainerStarted","Data":"7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890"} Sep 10 15:03:47.956845 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.956689 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" event={"ID":"f9d3a7d2-c726-40d8-b4ef-401446d587c2","Type":"ContainerStarted","Data":"476f1f4509eaba2c56fc102a479116c4742587a733554885705f61c978f55133"} Sep 10 15:03:47.956845 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.956774 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:03:47.979709 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:47.979654 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" podStartSLOduration=0.979633561 podStartE2EDuration="979.633561ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:03:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:03:47.976792842 +0000 UTC m=+1284.648818613" watchObservedRunningTime="2026-09-10 15:03:47.979633561 +0000 UTC m=+1284.651659336" Sep 10 15:03:58.964106 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:03:58.964070 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:04:28.739399 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:28.739367 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7"] Sep 10 15:04:28.739874 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:28.739681 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" podUID="ca05b596-7802-44bc-865f-26ceddd34f8c" containerName="maas-api" containerID="cri-o://41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd" gracePeriod=30 Sep 10 15:04:28.795938 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:28.795907 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5"] Sep 10 15:04:28.796216 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:28.796146 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" podUID="e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" containerName="payload-processing" containerID="cri-o://57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c" gracePeriod=30 Sep 10 15:04:28.803652 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:28.803050 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf"] Sep 10 15:04:28.803652 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:28.803290 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" podUID="b3e96518-e446-479a-ab46-571c6fb0d4ec" containerName="payload-pre-processing" containerID="cri-o://3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e" gracePeriod=30 Sep 10 15:04:29.094320 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.094295 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:04:29.114054 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.114022 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:04:29.117996 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.117973 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:04:29.156717 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.156688 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8j8v\" (UniqueName: \"kubernetes.io/projected/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-kube-api-access-s8j8v\") pod \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " Sep 10 15:04:29.156867 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.156821 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-plugins-config-volume\") pod \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\" (UID: \"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6\") " Sep 10 15:04:29.157195 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.157169 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" (UID: "e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:04:29.158880 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.158848 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-kube-api-access-s8j8v" (OuterVolumeSpecName: "kube-api-access-s8j8v") pod "e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" (UID: "e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6"). InnerVolumeSpecName "kube-api-access-s8j8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:04:29.165724 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.165701 2584 generic.go:356] "Generic (PLEG): container finished" podID="b3e96518-e446-479a-ab46-571c6fb0d4ec" containerID="3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e" exitCode=0 Sep 10 15:04:29.165844 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.165735 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" Sep 10 15:04:29.165844 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.165789 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" event={"ID":"b3e96518-e446-479a-ab46-571c6fb0d4ec","Type":"ContainerDied","Data":"3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e"} Sep 10 15:04:29.165964 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.165849 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf" event={"ID":"b3e96518-e446-479a-ab46-571c6fb0d4ec","Type":"ContainerDied","Data":"27124e2c8474d584cf7ca4f230e2d4666bbb5c36d4ce28cc75660b4336353e92"} Sep 10 15:04:29.165964 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.165882 2584 scope.go:117] "RemoveContainer" containerID="3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e" Sep 10 15:04:29.167261 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.167236 2584 generic.go:356] "Generic (PLEG): container finished" podID="e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" containerID="57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c" exitCode=0 Sep 10 15:04:29.167358 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.167325 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" Sep 10 15:04:29.167430 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.167379 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" event={"ID":"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6","Type":"ContainerDied","Data":"57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c"} Sep 10 15:04:29.167491 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.167442 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5" event={"ID":"e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6","Type":"ContainerDied","Data":"5e43ff2865dfdc8bc0a0b2816712972f9e2797461996e7dd71eaab89bd819770"} Sep 10 15:04:29.169016 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.168994 2584 generic.go:356] "Generic (PLEG): container finished" podID="ca05b596-7802-44bc-865f-26ceddd34f8c" containerID="41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd" exitCode=0 Sep 10 15:04:29.169111 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.169031 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" event={"ID":"ca05b596-7802-44bc-865f-26ceddd34f8c","Type":"ContainerDied","Data":"41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd"} Sep 10 15:04:29.169111 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.169036 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" Sep 10 15:04:29.169111 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.169067 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7" event={"ID":"ca05b596-7802-44bc-865f-26ceddd34f8c","Type":"ContainerDied","Data":"98fa6e8bed58698839299ace93c6626ee328c40ff94a39c41a44f467dfddd0cc"} Sep 10 15:04:29.175293 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.175274 2584 scope.go:117] "RemoveContainer" containerID="3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e" Sep 10 15:04:29.175590 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:04:29.175569 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e\": container with ID starting with 3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e not found: ID does not exist" containerID="3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e" Sep 10 15:04:29.175663 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.175602 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e"} err="failed to get container status \"3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e\": rpc error: code = NotFound desc = could not find container \"3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e\": container with ID starting with 3daa663e198783b87de6d851de623fd0bcf8912157daf6e34baf992c7feba63e not found: ID does not exist" Sep 10 15:04:29.175663 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.175624 2584 scope.go:117] "RemoveContainer" containerID="57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c" Sep 10 15:04:29.183972 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.183952 2584 scope.go:117] "RemoveContainer" containerID="57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c" Sep 10 15:04:29.184224 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:04:29.184192 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c\": container with ID starting with 57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c not found: ID does not exist" containerID="57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c" Sep 10 15:04:29.184278 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.184232 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c"} err="failed to get container status \"57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c\": rpc error: code = NotFound desc = could not find container \"57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c\": container with ID starting with 57c47cc719ec268a77098b259522d05eb3eaf408d44cae4af9b304b863d8d06c not found: ID does not exist" Sep 10 15:04:29.184278 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.184246 2584 scope.go:117] "RemoveContainer" containerID="41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd" Sep 10 15:04:29.192475 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.192459 2584 scope.go:117] "RemoveContainer" containerID="41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd" Sep 10 15:04:29.192715 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:04:29.192700 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd\": container with ID starting with 41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd not found: ID does not exist" containerID="41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd" Sep 10 15:04:29.192759 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.192722 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd"} err="failed to get container status \"41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd\": rpc error: code = NotFound desc = could not find container \"41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd\": container with ID starting with 41b5cc38d7fd82845523107bb2cee09c4b19c5f3e7a31ef93b3a674e6e88d8fd not found: ID does not exist" Sep 10 15:04:29.192908 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.192891 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5"] Sep 10 15:04:29.196329 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.196312 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-550dc3-66f8b776bc-t87b5"] Sep 10 15:04:29.258179 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258124 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqk55\" (UniqueName: \"kubernetes.io/projected/ca05b596-7802-44bc-865f-26ceddd34f8c-kube-api-access-sqk55\") pod \"ca05b596-7802-44bc-865f-26ceddd34f8c\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " Sep 10 15:04:29.258277 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258222 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca05b596-7802-44bc-865f-26ceddd34f8c-openshift-service-ca\") pod \"ca05b596-7802-44bc-865f-26ceddd34f8c\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " Sep 10 15:04:29.258326 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258300 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e96518-e446-479a-ab46-571c6fb0d4ec-plugins-config-volume\") pod \"b3e96518-e446-479a-ab46-571c6fb0d4ec\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " Sep 10 15:04:29.258383 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258365 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzq68\" (UniqueName: \"kubernetes.io/projected/b3e96518-e446-479a-ab46-571c6fb0d4ec-kube-api-access-wzq68\") pod \"b3e96518-e446-479a-ab46-571c6fb0d4ec\" (UID: \"b3e96518-e446-479a-ab46-571c6fb0d4ec\") " Sep 10 15:04:29.258447 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258401 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls\") pod \"ca05b596-7802-44bc-865f-26ceddd34f8c\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " Sep 10 15:04:29.258447 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258432 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-metrics-certs\") pod \"ca05b596-7802-44bc-865f-26ceddd34f8c\" (UID: \"ca05b596-7802-44bc-865f-26ceddd34f8c\") " Sep 10 15:04:29.258552 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258470 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca05b596-7802-44bc-865f-26ceddd34f8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca05b596-7802-44bc-865f-26ceddd34f8c" (UID: "ca05b596-7802-44bc-865f-26ceddd34f8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:04:29.258725 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258701 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e96518-e446-479a-ab46-571c6fb0d4ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3e96518-e446-479a-ab46-571c6fb0d4ec" (UID: "b3e96518-e446-479a-ab46-571c6fb0d4ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:04:29.258831 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258805 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca05b596-7802-44bc-865f-26ceddd34f8c-openshift-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.258831 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258826 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3e96518-e446-479a-ab46-571c6fb0d4ec-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.258953 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258841 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.258953 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.258854 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8j8v\" (UniqueName: \"kubernetes.io/projected/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6-kube-api-access-s8j8v\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.260824 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.260798 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ca05b596-7802-44bc-865f-26ceddd34f8c" (UID: "ca05b596-7802-44bc-865f-26ceddd34f8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:04:29.260824 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.260809 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3e96518-e446-479a-ab46-571c6fb0d4ec-kube-api-access-wzq68" (OuterVolumeSpecName: "kube-api-access-wzq68") pod "b3e96518-e446-479a-ab46-571c6fb0d4ec" (UID: "b3e96518-e446-479a-ab46-571c6fb0d4ec"). InnerVolumeSpecName "kube-api-access-wzq68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:04:29.260958 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.260843 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca05b596-7802-44bc-865f-26ceddd34f8c-kube-api-access-sqk55" (OuterVolumeSpecName: "kube-api-access-sqk55") pod "ca05b596-7802-44bc-865f-26ceddd34f8c" (UID: "ca05b596-7802-44bc-865f-26ceddd34f8c"). InnerVolumeSpecName "kube-api-access-sqk55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:04:29.261001 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.260965 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca05b596-7802-44bc-865f-26ceddd34f8c" (UID: "ca05b596-7802-44bc-865f-26ceddd34f8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:04:29.360139 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.360118 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqk55\" (UniqueName: \"kubernetes.io/projected/ca05b596-7802-44bc-865f-26ceddd34f8c-kube-api-access-sqk55\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.360139 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.360136 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzq68\" (UniqueName: \"kubernetes.io/projected/b3e96518-e446-479a-ab46-571c6fb0d4ec-kube-api-access-wzq68\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.360307 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.360148 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-maas-api-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.360307 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.360157 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca05b596-7802-44bc-865f-26ceddd34f8c-metrics-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:29.493705 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.493678 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf"] Sep 10 15:04:29.496731 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.496706 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-550dc3-688c8fc4c8-74lxf"] Sep 10 15:04:29.508768 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.508717 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7"] Sep 10 15:04:29.513745 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.513724 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-550dc3-7cd486fdc9-vw8j7"] Sep 10 15:04:29.763851 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.763789 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3e96518-e446-479a-ab46-571c6fb0d4ec" path="/var/lib/kubelet/pods/b3e96518-e446-479a-ab46-571c6fb0d4ec/volumes" Sep 10 15:04:29.764168 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.764154 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca05b596-7802-44bc-865f-26ceddd34f8c" path="/var/lib/kubelet/pods/ca05b596-7802-44bc-865f-26ceddd34f8c/volumes" Sep 10 15:04:29.764486 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:29.764474 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" path="/var/lib/kubelet/pods/e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6/volumes" Sep 10 15:04:35.782906 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.782873 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg"] Sep 10 15:04:35.783372 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783262 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca05b596-7802-44bc-865f-26ceddd34f8c" containerName="maas-api" Sep 10 15:04:35.783372 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783274 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca05b596-7802-44bc-865f-26ceddd34f8c" containerName="maas-api" Sep 10 15:04:35.783372 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783285 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3e96518-e446-479a-ab46-571c6fb0d4ec" containerName="payload-pre-processing" Sep 10 15:04:35.783372 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783291 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3e96518-e446-479a-ab46-571c6fb0d4ec" containerName="payload-pre-processing" Sep 10 15:04:35.783372 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783299 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" containerName="payload-processing" Sep 10 15:04:35.783372 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783304 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" containerName="payload-processing" Sep 10 15:04:35.783372 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783371 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca05b596-7802-44bc-865f-26ceddd34f8c" containerName="maas-api" Sep 10 15:04:35.783593 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783381 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3e96518-e446-479a-ab46-571c6fb0d4ec" containerName="payload-pre-processing" Sep 10 15:04:35.783593 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.783387 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1b3bb9f-a11c-435e-82d2-f5c27b0fadc6" containerName="payload-processing" Sep 10 15:04:35.788002 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.787985 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" Sep 10 15:04:35.790952 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.790933 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\"" Sep 10 15:04:35.791258 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.791245 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:04:35.797846 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.797825 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg"] Sep 10 15:04:35.914319 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:35.914288 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a331ea79-4d55-48db-ae42-b4f080894b8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg\" (UID: \"a331ea79-4d55-48db-ae42-b4f080894b8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" Sep 10 15:04:36.016406 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:36.016371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a331ea79-4d55-48db-ae42-b4f080894b8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg\" (UID: \"a331ea79-4d55-48db-ae42-b4f080894b8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" Sep 10 15:04:36.016956 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:36.016935 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a331ea79-4d55-48db-ae42-b4f080894b8b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg\" (UID: \"a331ea79-4d55-48db-ae42-b4f080894b8b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" Sep 10 15:04:36.099078 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:36.099015 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" Sep 10 15:04:36.370739 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:36.370650 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg"] Sep 10 15:04:36.375294 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:04:36.375263 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda331ea79_4d55_48db_ae42_b4f080894b8b.slice/crio-2d783ffa3751a355c9c6741e8836e3e171fc4ccb63980d3aebc805af3b90cc72 WatchSource:0}: Error finding container 2d783ffa3751a355c9c6741e8836e3e171fc4ccb63980d3aebc805af3b90cc72: Status 404 returned error can't find the container with id 2d783ffa3751a355c9c6741e8836e3e171fc4ccb63980d3aebc805af3b90cc72 Sep 10 15:04:37.215245 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:37.215185 2584 generic.go:356] "Generic (PLEG): container finished" podID="a331ea79-4d55-48db-ae42-b4f080894b8b" containerID="380f7390d59109a3a696ffaeaddd9fedae533dd21388f1f795676739cc691218" exitCode=0 Sep 10 15:04:37.215571 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:37.215276 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" event={"ID":"a331ea79-4d55-48db-ae42-b4f080894b8b","Type":"ContainerDied","Data":"380f7390d59109a3a696ffaeaddd9fedae533dd21388f1f795676739cc691218"} Sep 10 15:04:37.215571 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:37.215325 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" event={"ID":"a331ea79-4d55-48db-ae42-b4f080894b8b","Type":"ContainerStarted","Data":"2d783ffa3751a355c9c6741e8836e3e171fc4ccb63980d3aebc805af3b90cc72"} Sep 10 15:04:38.349560 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:38.349537 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" Sep 10 15:04:38.436177 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:38.436150 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a331ea79-4d55-48db-ae42-b4f080894b8b-maas-api-service-ca\") pod \"a331ea79-4d55-48db-ae42-b4f080894b8b\" (UID: \"a331ea79-4d55-48db-ae42-b4f080894b8b\") " Sep 10 15:04:38.436490 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:38.436469 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a331ea79-4d55-48db-ae42-b4f080894b8b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a331ea79-4d55-48db-ae42-b4f080894b8b" (UID: "a331ea79-4d55-48db-ae42-b4f080894b8b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:04:38.537444 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:38.537382 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a331ea79-4d55-48db-ae42-b4f080894b8b-maas-api-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:39.226631 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:39.226597 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" event={"ID":"a331ea79-4d55-48db-ae42-b4f080894b8b","Type":"ContainerDied","Data":"2d783ffa3751a355c9c6741e8836e3e171fc4ccb63980d3aebc805af3b90cc72"} Sep 10 15:04:39.226796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:39.226651 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d783ffa3751a355c9c6741e8836e3e171fc4ccb63980d3aebc805af3b90cc72" Sep 10 15:04:39.226796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:39.226615 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg" Sep 10 15:04:51.454648 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.454295 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh"] Sep 10 15:04:51.455193 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.454636 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" podUID="f9d3a7d2-c726-40d8-b4ef-401446d587c2" containerName="payload-pre-processing" containerID="cri-o://7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890" gracePeriod=30 Sep 10 15:04:51.626059 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.626029 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg"] Sep 10 15:04:51.628841 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.628815 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-cb65dc-j9xfg"] Sep 10 15:04:51.719140 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.719055 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:04:51.763825 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.763790 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a331ea79-4d55-48db-ae42-b4f080894b8b" path="/var/lib/kubelet/pods/a331ea79-4d55-48db-ae42-b4f080894b8b/volumes" Sep 10 15:04:51.841076 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.841049 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5s99\" (UniqueName: \"kubernetes.io/projected/f9d3a7d2-c726-40d8-b4ef-401446d587c2-kube-api-access-s5s99\") pod \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " Sep 10 15:04:51.841253 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.841130 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d3a7d2-c726-40d8-b4ef-401446d587c2-plugins-config-volume\") pod \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\" (UID: \"f9d3a7d2-c726-40d8-b4ef-401446d587c2\") " Sep 10 15:04:51.841502 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.841481 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9d3a7d2-c726-40d8-b4ef-401446d587c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9d3a7d2-c726-40d8-b4ef-401446d587c2" (UID: "f9d3a7d2-c726-40d8-b4ef-401446d587c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:04:51.843126 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.843095 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d3a7d2-c726-40d8-b4ef-401446d587c2-kube-api-access-s5s99" (OuterVolumeSpecName: "kube-api-access-s5s99") pod "f9d3a7d2-c726-40d8-b4ef-401446d587c2" (UID: "f9d3a7d2-c726-40d8-b4ef-401446d587c2"). InnerVolumeSpecName "kube-api-access-s5s99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:04:51.941735 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.941710 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5s99\" (UniqueName: \"kubernetes.io/projected/f9d3a7d2-c726-40d8-b4ef-401446d587c2-kube-api-access-s5s99\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:51.941735 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:51.941733 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9d3a7d2-c726-40d8-b4ef-401446d587c2-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:04:52.289966 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.289932 2584 generic.go:356] "Generic (PLEG): container finished" podID="f9d3a7d2-c726-40d8-b4ef-401446d587c2" containerID="7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890" exitCode=0 Sep 10 15:04:52.290129 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.289984 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" Sep 10 15:04:52.290129 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.289984 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" event={"ID":"f9d3a7d2-c726-40d8-b4ef-401446d587c2","Type":"ContainerDied","Data":"7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890"} Sep 10 15:04:52.290129 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.290048 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh" event={"ID":"f9d3a7d2-c726-40d8-b4ef-401446d587c2","Type":"ContainerDied","Data":"476f1f4509eaba2c56fc102a479116c4742587a733554885705f61c978f55133"} Sep 10 15:04:52.290129 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.290078 2584 scope.go:117] "RemoveContainer" containerID="7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890" Sep 10 15:04:52.300534 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.300517 2584 scope.go:117] "RemoveContainer" containerID="7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890" Sep 10 15:04:52.300831 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:04:52.300810 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890\": container with ID starting with 7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890 not found: ID does not exist" containerID="7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890" Sep 10 15:04:52.300932 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.300838 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890"} err="failed to get container status \"7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890\": rpc error: code = NotFound desc = could not find container \"7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890\": container with ID starting with 7fbea3ed312b4949a23b2e9078cb0cb0ee485ea6dccfed9688594139a9442890 not found: ID does not exist" Sep 10 15:04:52.314990 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.314971 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh"] Sep 10 15:04:52.327623 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:52.327600 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-cb65dc-684cfbbdb6-t7ghh"] Sep 10 15:04:53.762711 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:53.762676 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9d3a7d2-c726-40d8-b4ef-401446d587c2" path="/var/lib/kubelet/pods/f9d3a7d2-c726-40d8-b4ef-401446d587c2/volumes" Sep 10 15:04:58.469930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.467082 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr"] Sep 10 15:04:58.469930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.467906 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a331ea79-4d55-48db-ae42-b4f080894b8b" containerName="revoke-keys" Sep 10 15:04:58.469930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.467927 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="a331ea79-4d55-48db-ae42-b4f080894b8b" containerName="revoke-keys" Sep 10 15:04:58.469930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.467969 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9d3a7d2-c726-40d8-b4ef-401446d587c2" containerName="payload-pre-processing" Sep 10 15:04:58.469930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.467977 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9d3a7d2-c726-40d8-b4ef-401446d587c2" containerName="payload-pre-processing" Sep 10 15:04:58.469930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.468146 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="a331ea79-4d55-48db-ae42-b4f080894b8b" containerName="revoke-keys" Sep 10 15:04:58.469930 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.468158 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9d3a7d2-c726-40d8-b4ef-401446d587c2" containerName="payload-pre-processing" Sep 10 15:04:58.472220 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.472168 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" Sep 10 15:04:58.481963 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.481909 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\"" Sep 10 15:04:58.481963 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.481958 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:04:58.483399 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.483378 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr"] Sep 10 15:04:58.595501 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.595473 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da160b5-ab60-49c3-83ef-c851d3a8b8bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr\" (UID: \"9da160b5-ab60-49c3-83ef-c851d3a8b8bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" Sep 10 15:04:58.696126 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.696092 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da160b5-ab60-49c3-83ef-c851d3a8b8bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr\" (UID: \"9da160b5-ab60-49c3-83ef-c851d3a8b8bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" Sep 10 15:04:58.696697 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.696676 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da160b5-ab60-49c3-83ef-c851d3a8b8bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr\" (UID: \"9da160b5-ab60-49c3-83ef-c851d3a8b8bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" Sep 10 15:04:58.782131 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:58.782074 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" Sep 10 15:04:59.057585 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:59.057558 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr"] Sep 10 15:04:59.059479 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:04:59.059446 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da160b5_ab60_49c3_83ef_c851d3a8b8bb.slice/crio-276ab08b0abd466d69bbccd775bd189fc7a72a37d50ac3f4eb5c7c48491e7ded WatchSource:0}: Error finding container 276ab08b0abd466d69bbccd775bd189fc7a72a37d50ac3f4eb5c7c48491e7ded: Status 404 returned error can't find the container with id 276ab08b0abd466d69bbccd775bd189fc7a72a37d50ac3f4eb5c7c48491e7ded Sep 10 15:04:59.331840 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:04:59.331754 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" event={"ID":"9da160b5-ab60-49c3-83ef-c851d3a8b8bb","Type":"ContainerStarted","Data":"276ab08b0abd466d69bbccd775bd189fc7a72a37d50ac3f4eb5c7c48491e7ded"} Sep 10 15:05:00.338265 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:00.338227 2584 generic.go:356] "Generic (PLEG): container finished" podID="9da160b5-ab60-49c3-83ef-c851d3a8b8bb" containerID="f18052aef2d64ebd1e408183ee834d4cf5fff8d619e47b8205ea53867e50295f" exitCode=0 Sep 10 15:05:00.338638 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:00.338291 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" event={"ID":"9da160b5-ab60-49c3-83ef-c851d3a8b8bb","Type":"ContainerDied","Data":"f18052aef2d64ebd1e408183ee834d4cf5fff8d619e47b8205ea53867e50295f"} Sep 10 15:05:01.482754 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:01.482725 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" Sep 10 15:05:01.521954 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:01.521928 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da160b5-ab60-49c3-83ef-c851d3a8b8bb-maas-api-service-ca\") pod \"9da160b5-ab60-49c3-83ef-c851d3a8b8bb\" (UID: \"9da160b5-ab60-49c3-83ef-c851d3a8b8bb\") " Sep 10 15:05:01.522298 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:01.522277 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9da160b5-ab60-49c3-83ef-c851d3a8b8bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9da160b5-ab60-49c3-83ef-c851d3a8b8bb" (UID: "9da160b5-ab60-49c3-83ef-c851d3a8b8bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:05:01.623550 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:01.623464 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da160b5-ab60-49c3-83ef-c851d3a8b8bb-maas-api-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:05:02.349045 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:02.349013 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" Sep 10 15:05:02.349245 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:02.349043 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr" event={"ID":"9da160b5-ab60-49c3-83ef-c851d3a8b8bb","Type":"ContainerDied","Data":"276ab08b0abd466d69bbccd775bd189fc7a72a37d50ac3f4eb5c7c48491e7ded"} Sep 10 15:05:02.349245 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:02.349089 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="276ab08b0abd466d69bbccd775bd189fc7a72a37d50ac3f4eb5c7c48491e7ded" Sep 10 15:05:09.439044 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:09.438993 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr"] Sep 10 15:05:09.441723 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:09.441698 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-5fb994-r2pwr"] Sep 10 15:05:09.763216 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:09.763161 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9da160b5-ab60-49c3-83ef-c851d3a8b8bb" path="/var/lib/kubelet/pods/9da160b5-ab60-49c3-83ef-c851d3a8b8bb/volumes" Sep 10 15:05:26.914229 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:26.914181 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln"] Sep 10 15:05:26.914691 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:26.914456 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" podUID="ccc11795-fc2f-4b37-be41-052d2abae239" containerName="payload-processing" containerID="cri-o://e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c" gracePeriod=30 Sep 10 15:05:26.924461 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:26.924433 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5"] Sep 10 15:05:26.924691 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:26.924666 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" podUID="12336108-5b25-4a87-b4a3-f2f65d9c3c4b" containerName="payload-pre-processing" containerID="cri-o://a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852" gracePeriod=30 Sep 10 15:05:27.205846 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.205821 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 15:05:27.208433 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.208415 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 15:05:27.357278 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.357250 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-plugins-config-volume\") pod \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " Sep 10 15:05:27.357449 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.357306 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l49r2\" (UniqueName: \"kubernetes.io/projected/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-kube-api-access-l49r2\") pod \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\" (UID: \"12336108-5b25-4a87-b4a3-f2f65d9c3c4b\") " Sep 10 15:05:27.357449 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.357385 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc11795-fc2f-4b37-be41-052d2abae239-plugins-config-volume\") pod \"ccc11795-fc2f-4b37-be41-052d2abae239\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " Sep 10 15:05:27.357449 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.357444 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl9kw\" (UniqueName: \"kubernetes.io/projected/ccc11795-fc2f-4b37-be41-052d2abae239-kube-api-access-hl9kw\") pod \"ccc11795-fc2f-4b37-be41-052d2abae239\" (UID: \"ccc11795-fc2f-4b37-be41-052d2abae239\") " Sep 10 15:05:27.357740 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.357715 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccc11795-fc2f-4b37-be41-052d2abae239-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ccc11795-fc2f-4b37-be41-052d2abae239" (UID: "ccc11795-fc2f-4b37-be41-052d2abae239"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:05:27.357830 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.357719 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12336108-5b25-4a87-b4a3-f2f65d9c3c4b" (UID: "12336108-5b25-4a87-b4a3-f2f65d9c3c4b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:05:27.359424 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.359398 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-kube-api-access-l49r2" (OuterVolumeSpecName: "kube-api-access-l49r2") pod "12336108-5b25-4a87-b4a3-f2f65d9c3c4b" (UID: "12336108-5b25-4a87-b4a3-f2f65d9c3c4b"). InnerVolumeSpecName "kube-api-access-l49r2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:05:27.359594 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.359572 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccc11795-fc2f-4b37-be41-052d2abae239-kube-api-access-hl9kw" (OuterVolumeSpecName: "kube-api-access-hl9kw") pod "ccc11795-fc2f-4b37-be41-052d2abae239" (UID: "ccc11795-fc2f-4b37-be41-052d2abae239"). InnerVolumeSpecName "kube-api-access-hl9kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:05:27.458583 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.458559 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc11795-fc2f-4b37-be41-052d2abae239-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:05:27.458583 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.458580 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl9kw\" (UniqueName: \"kubernetes.io/projected/ccc11795-fc2f-4b37-be41-052d2abae239-kube-api-access-hl9kw\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:05:27.458721 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.458589 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:05:27.458721 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.458598 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l49r2\" (UniqueName: \"kubernetes.io/projected/12336108-5b25-4a87-b4a3-f2f65d9c3c4b-kube-api-access-l49r2\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:05:27.478379 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.478349 2584 generic.go:356] "Generic (PLEG): container finished" podID="ccc11795-fc2f-4b37-be41-052d2abae239" containerID="e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c" exitCode=0 Sep 10 15:05:27.478494 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.478385 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" Sep 10 15:05:27.478494 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.478424 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" event={"ID":"ccc11795-fc2f-4b37-be41-052d2abae239","Type":"ContainerDied","Data":"e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c"} Sep 10 15:05:27.478494 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.478470 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln" event={"ID":"ccc11795-fc2f-4b37-be41-052d2abae239","Type":"ContainerDied","Data":"7497d8d1cbd93e64e2ed89f364c88d53cb5ab1cd8e03ba8b64e77e9c8c4484a5"} Sep 10 15:05:27.478494 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.478494 2584 scope.go:117] "RemoveContainer" containerID="e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c" Sep 10 15:05:27.479922 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.479899 2584 generic.go:356] "Generic (PLEG): container finished" podID="12336108-5b25-4a87-b4a3-f2f65d9c3c4b" containerID="a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852" exitCode=0 Sep 10 15:05:27.480012 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.479933 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" event={"ID":"12336108-5b25-4a87-b4a3-f2f65d9c3c4b","Type":"ContainerDied","Data":"a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852"} Sep 10 15:05:27.480012 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.479955 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" Sep 10 15:05:27.480012 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.479986 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5" event={"ID":"12336108-5b25-4a87-b4a3-f2f65d9c3c4b","Type":"ContainerDied","Data":"cf6cc030f1bc594372586d0e698edd837f271642e0443e3a65380620401b158d"} Sep 10 15:05:27.486840 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.486824 2584 scope.go:117] "RemoveContainer" containerID="e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c" Sep 10 15:05:27.487105 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:05:27.487086 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c\": container with ID starting with e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c not found: ID does not exist" containerID="e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c" Sep 10 15:05:27.487168 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.487112 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c"} err="failed to get container status \"e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c\": rpc error: code = NotFound desc = could not find container \"e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c\": container with ID starting with e0a2c4ca1c53a5041b73c04e15ad9281bf2640f40d11bc4543251bd83498f75c not found: ID does not exist" Sep 10 15:05:27.487168 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.487127 2584 scope.go:117] "RemoveContainer" containerID="a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852" Sep 10 15:05:27.494924 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.494905 2584 scope.go:117] "RemoveContainer" containerID="a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852" Sep 10 15:05:27.495168 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:05:27.495150 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852\": container with ID starting with a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852 not found: ID does not exist" containerID="a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852" Sep 10 15:05:27.495237 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.495173 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852"} err="failed to get container status \"a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852\": rpc error: code = NotFound desc = could not find container \"a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852\": container with ID starting with a8dfc8d4252cea96363b35dfc109cfbec181e06e389059b8cf86816d31767852 not found: ID does not exist" Sep 10 15:05:27.507438 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.507416 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5"] Sep 10 15:05:27.511898 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.511877 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5899e8-8cd6b6d6-9rnd5"] Sep 10 15:05:27.523569 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.523549 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln"] Sep 10 15:05:27.527900 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.527882 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5899e8-7f5ff78cf9-27sln"] Sep 10 15:05:27.762589 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.762522 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12336108-5b25-4a87-b4a3-f2f65d9c3c4b" path="/var/lib/kubelet/pods/12336108-5b25-4a87-b4a3-f2f65d9c3c4b/volumes" Sep 10 15:05:27.762861 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:05:27.762848 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccc11795-fc2f-4b37-be41-052d2abae239" path="/var/lib/kubelet/pods/ccc11795-fc2f-4b37-be41-052d2abae239/volumes" Sep 10 15:06:44.675122 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.674620 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl"] Sep 10 15:06:44.676933 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.676905 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9da160b5-ab60-49c3-83ef-c851d3a8b8bb" containerName="revoke-keys" Sep 10 15:06:44.677098 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677085 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9da160b5-ab60-49c3-83ef-c851d3a8b8bb" containerName="revoke-keys" Sep 10 15:06:44.677230 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677218 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12336108-5b25-4a87-b4a3-f2f65d9c3c4b" containerName="payload-pre-processing" Sep 10 15:06:44.677315 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677307 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="12336108-5b25-4a87-b4a3-f2f65d9c3c4b" containerName="payload-pre-processing" Sep 10 15:06:44.677417 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677407 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccc11795-fc2f-4b37-be41-052d2abae239" containerName="payload-processing" Sep 10 15:06:44.677489 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677479 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccc11795-fc2f-4b37-be41-052d2abae239" containerName="payload-processing" Sep 10 15:06:44.677768 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677754 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccc11795-fc2f-4b37-be41-052d2abae239" containerName="payload-processing" Sep 10 15:06:44.677862 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677852 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9da160b5-ab60-49c3-83ef-c851d3a8b8bb" containerName="revoke-keys" Sep 10 15:06:44.677951 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.677941 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="12336108-5b25-4a87-b4a3-f2f65d9c3c4b" containerName="payload-pre-processing" Sep 10 15:06:44.684076 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.684057 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.689115 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.689085 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl"] Sep 10 15:06:44.689895 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.689871 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-b022ee-serving-cert\"" Sep 10 15:06:44.690085 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.689894 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9t8qm\"" Sep 10 15:06:44.691771 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.691753 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:06:44.692119 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.692102 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 15:06:44.692623 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.692605 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 15:06:44.751348 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.751310 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81233af0-d7e0-40ae-8800-fcbdb159e949-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.751513 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.751352 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.751513 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.751462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-metrics-certs\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.751513 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.751505 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khl4c\" (UniqueName: \"kubernetes.io/projected/81233af0-d7e0-40ae-8800-fcbdb159e949-kube-api-access-khl4c\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.852702 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.852661 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-metrics-certs\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.852910 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.852716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khl4c\" (UniqueName: \"kubernetes.io/projected/81233af0-d7e0-40ae-8800-fcbdb159e949-kube-api-access-khl4c\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.852910 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.852776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81233af0-d7e0-40ae-8800-fcbdb159e949-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.852910 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.852802 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.853087 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:06:44.852931 2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-serving-cert: secret "maas-api-e2e-ipp-a-b022ee-serving-cert" not found Sep 10 15:06:44.853087 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:06:44.853001 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls podName:81233af0-d7e0-40ae-8800-fcbdb159e949 nodeName:}" failed. No retries permitted until 2026-09-10 15:06:45.352980818 +0000 UTC m=+1462.025006570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls") pod "maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" (UID: "81233af0-d7e0-40ae-8800-fcbdb159e949") : secret "maas-api-e2e-ipp-a-b022ee-serving-cert" not found Sep 10 15:06:44.853522 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.853499 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81233af0-d7e0-40ae-8800-fcbdb159e949-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.855487 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.855459 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-metrics-certs\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:44.864004 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:44.863980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khl4c\" (UniqueName: \"kubernetes.io/projected/81233af0-d7e0-40ae-8800-fcbdb159e949-kube-api-access-khl4c\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:45.358358 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:45.358319 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:45.360713 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:45.360683 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls\") pod \"maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:45.600381 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:45.600347 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:45.872847 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:45.872761 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl"] Sep 10 15:06:45.876317 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:06:45.876288 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81233af0_d7e0_40ae_8800_fcbdb159e949.slice/crio-7c848e620cdb6b85351a343ea503ba419562cf4513b6fef47934e24feeac3885 WatchSource:0}: Error finding container 7c848e620cdb6b85351a343ea503ba419562cf4513b6fef47934e24feeac3885: Status 404 returned error can't find the container with id 7c848e620cdb6b85351a343ea503ba419562cf4513b6fef47934e24feeac3885 Sep 10 15:06:45.877564 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:45.877544 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 15:06:46.868886 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:46.868851 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" event={"ID":"81233af0-d7e0-40ae-8800-fcbdb159e949","Type":"ContainerStarted","Data":"8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f"} Sep 10 15:06:46.869070 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:46.868904 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" event={"ID":"81233af0-d7e0-40ae-8800-fcbdb159e949","Type":"ContainerStarted","Data":"7c848e620cdb6b85351a343ea503ba419562cf4513b6fef47934e24feeac3885"} Sep 10 15:06:46.869070 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:46.868986 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:06:46.890515 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:46.890461 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" podStartSLOduration=2.4872950019999998 podStartE2EDuration="2.89044513s" totalImagesPullingTime="403.150128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:06:46.887565904 +0000 UTC m=+1463.559591677" watchObservedRunningTime="2026-09-10 15:06:46.89044513 +0000 UTC m=+1463.562470916" Sep 10 15:06:52.879612 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:06:52.879582 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:07:14.735697 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.735653 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67"] Sep 10 15:07:14.740068 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.740033 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:14.743326 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.743300 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-cb5c8c-dockercfg-rkbwq\"" Sep 10 15:07:14.743640 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.743618 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-cb5c8c\"" Sep 10 15:07:14.764942 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.763261 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67"] Sep 10 15:07:14.831015 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.830967 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68b6m\" (UniqueName: \"kubernetes.io/projected/02548f5b-563d-4965-964f-658db34581ce-kube-api-access-68b6m\") pod \"payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:14.831223 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.831051 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02548f5b-563d-4965-964f-658db34581ce-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:14.932095 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.932061 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68b6m\" (UniqueName: \"kubernetes.io/projected/02548f5b-563d-4965-964f-658db34581ce-kube-api-access-68b6m\") pod \"payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:14.932095 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.932099 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02548f5b-563d-4965-964f-658db34581ce-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:14.932760 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.932741 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02548f5b-563d-4965-964f-658db34581ce-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:14.942978 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:14.942950 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68b6m\" (UniqueName: \"kubernetes.io/projected/02548f5b-563d-4965-964f-658db34581ce-kube-api-access-68b6m\") pod \"payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:15.054471 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:15.054378 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:15.331564 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:15.331537 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67"] Sep 10 15:07:15.333269 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:07:15.333235 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02548f5b_563d_4965_964f_658db34581ce.slice/crio-5c48160218685ba2aff6a5fcd3df88e539eb4c4d05f8c42e3ceec493b554438d WatchSource:0}: Error finding container 5c48160218685ba2aff6a5fcd3df88e539eb4c4d05f8c42e3ceec493b554438d: Status 404 returned error can't find the container with id 5c48160218685ba2aff6a5fcd3df88e539eb4c4d05f8c42e3ceec493b554438d Sep 10 15:07:16.011993 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:16.011955 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" event={"ID":"02548f5b-563d-4965-964f-658db34581ce","Type":"ContainerStarted","Data":"a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545"} Sep 10 15:07:16.012434 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:16.012016 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" event={"ID":"02548f5b-563d-4965-964f-658db34581ce","Type":"ContainerStarted","Data":"5c48160218685ba2aff6a5fcd3df88e539eb4c4d05f8c42e3ceec493b554438d"} Sep 10 15:07:16.012434 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:16.012167 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:16.034922 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:16.034868 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" podStartSLOduration=2.034849927 podStartE2EDuration="2.034849927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:07:16.03081421 +0000 UTC m=+1492.702839984" watchObservedRunningTime="2026-09-10 15:07:16.034849927 +0000 UTC m=+1492.706875701" Sep 10 15:07:23.816270 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:23.815958 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:07:23.827777 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:23.827754 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:07:27.019971 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:27.019939 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:07:42.704975 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:42.704944 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scfq4_e92eeed2-5be2-46ad-8c24-65ef2f89b0c8/payload-processing/0.log" Sep 10 15:07:42.821904 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:07:42.821874 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67_02548f5b-563d-4965-964f-658db34581ce/payload-processing/0.log" Sep 10 15:08:49.332711 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:49.332673 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scfq4_e92eeed2-5be2-46ad-8c24-65ef2f89b0c8/payload-processing/0.log" Sep 10 15:08:49.447712 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:49.447684 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67_02548f5b-563d-4965-964f-658db34581ce/payload-processing/0.log" Sep 10 15:08:54.271595 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.271553 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd"] Sep 10 15:08:54.275665 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.275638 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.279006 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.278983 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-799b5a-serving-cert\"" Sep 10 15:08:54.290668 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.290639 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd"] Sep 10 15:08:54.341117 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.341083 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.341337 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.341122 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c911aae3-876b-4ab6-96bb-f72a95936a5d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.341337 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.341160 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwr5q\" (UniqueName: \"kubernetes.io/projected/c911aae3-876b-4ab6-96bb-f72a95936a5d-kube-api-access-lwr5q\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.341337 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.341287 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.442561 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.442529 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.442782 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.442588 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.442782 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.442610 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c911aae3-876b-4ab6-96bb-f72a95936a5d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.442782 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.442641 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwr5q\" (UniqueName: \"kubernetes.io/projected/c911aae3-876b-4ab6-96bb-f72a95936a5d-kube-api-access-lwr5q\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.442782 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:08:54.442728 2584 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-serving-cert: secret "maas-api-e2e-ipp-cleanup-799b5a-serving-cert" not found Sep 10 15:08:54.443024 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:08:54.442809 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls podName:c911aae3-876b-4ab6-96bb-f72a95936a5d nodeName:}" failed. No retries permitted until 2026-09-10 15:08:54.942787317 +0000 UTC m=+1591.614813078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" (UID: "c911aae3-876b-4ab6-96bb-f72a95936a5d") : secret "maas-api-e2e-ipp-cleanup-799b5a-serving-cert" not found Sep 10 15:08:54.443329 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.443310 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c911aae3-876b-4ab6-96bb-f72a95936a5d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.445054 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.445036 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.453701 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.453680 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwr5q\" (UniqueName: \"kubernetes.io/projected/c911aae3-876b-4ab6-96bb-f72a95936a5d-kube-api-access-lwr5q\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.948609 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.948573 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:54.950860 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:54.950837 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:55.027532 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.027494 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6"] Sep 10 15:08:55.030938 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.030922 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.034158 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.034133 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-799b5a-dockercfg-cwhzt\"" Sep 10 15:08:55.034321 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.034187 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-799b5a\"" Sep 10 15:08:55.045082 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.045047 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6"] Sep 10 15:08:55.150350 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.150310 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69349758-fe24-4a76-88fe-40f0f7b9e97f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.150535 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.150370 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szwn4\" (UniqueName: \"kubernetes.io/projected/69349758-fe24-4a76-88fe-40f0f7b9e97f-kube-api-access-szwn4\") pod \"payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.189153 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.189122 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:55.251667 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.251187 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69349758-fe24-4a76-88fe-40f0f7b9e97f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.251667 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.251272 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szwn4\" (UniqueName: \"kubernetes.io/projected/69349758-fe24-4a76-88fe-40f0f7b9e97f-kube-api-access-szwn4\") pod \"payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.252003 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.251980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69349758-fe24-4a76-88fe-40f0f7b9e97f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.260790 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.260761 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szwn4\" (UniqueName: \"kubernetes.io/projected/69349758-fe24-4a76-88fe-40f0f7b9e97f-kube-api-access-szwn4\") pod \"payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.342180 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.342141 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:55.462231 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.462194 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd"] Sep 10 15:08:55.463796 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:08:55.463765 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc911aae3_876b_4ab6_96bb_f72a95936a5d.slice/crio-adf06916d73bc5bf938198df8df2c42c20a6639405e18acc66c6e863c744392d WatchSource:0}: Error finding container adf06916d73bc5bf938198df8df2c42c20a6639405e18acc66c6e863c744392d: Status 404 returned error can't find the container with id adf06916d73bc5bf938198df8df2c42c20a6639405e18acc66c6e863c744392d Sep 10 15:08:55.505391 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.505359 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" event={"ID":"c911aae3-876b-4ab6-96bb-f72a95936a5d","Type":"ContainerStarted","Data":"adf06916d73bc5bf938198df8df2c42c20a6639405e18acc66c6e863c744392d"} Sep 10 15:08:55.614810 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:55.614787 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6"] Sep 10 15:08:55.616731 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:08:55.616697 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69349758_fe24_4a76_88fe_40f0f7b9e97f.slice/crio-d4ec4c91c185cd969c23f52c9196ad1880dfa2296b6ad07faf015f08975299e4 WatchSource:0}: Error finding container d4ec4c91c185cd969c23f52c9196ad1880dfa2296b6ad07faf015f08975299e4: Status 404 returned error can't find the container with id d4ec4c91c185cd969c23f52c9196ad1880dfa2296b6ad07faf015f08975299e4 Sep 10 15:08:56.512446 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:56.512402 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" event={"ID":"69349758-fe24-4a76-88fe-40f0f7b9e97f","Type":"ContainerStarted","Data":"ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98"} Sep 10 15:08:56.512897 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:56.512558 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:08:56.512897 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:56.512593 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" event={"ID":"69349758-fe24-4a76-88fe-40f0f7b9e97f","Type":"ContainerStarted","Data":"d4ec4c91c185cd969c23f52c9196ad1880dfa2296b6ad07faf015f08975299e4"} Sep 10 15:08:56.513951 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:56.513926 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" event={"ID":"c911aae3-876b-4ab6-96bb-f72a95936a5d","Type":"ContainerStarted","Data":"6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8"} Sep 10 15:08:56.514087 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:56.514075 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:08:56.529989 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:56.529946 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" podStartSLOduration=1.529931428 podStartE2EDuration="1.529931428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:08:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:08:56.528650479 +0000 UTC m=+1593.200676252" watchObservedRunningTime="2026-09-10 15:08:56.529931428 +0000 UTC m=+1593.201957199" Sep 10 15:08:56.546697 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:08:56.546646 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" podStartSLOduration=2.185760799 podStartE2EDuration="2.546631064s" totalImagesPullingTime="360.870265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:08:56.545141771 +0000 UTC m=+1593.217167542" watchObservedRunningTime="2026-09-10 15:08:56.546631064 +0000 UTC m=+1593.218656873" Sep 10 15:09:02.525622 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:02.525593 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:09:07.522300 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:07.522261 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:09:24.859175 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:24.859091 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd"] Sep 10 15:09:24.859753 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:24.859368 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" podUID="c911aae3-876b-4ab6-96bb-f72a95936a5d" containerName="maas-api" containerID="cri-o://6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8" gracePeriod=30 Sep 10 15:09:24.939376 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:24.939343 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6"] Sep 10 15:09:24.939735 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:24.939556 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" podUID="69349758-fe24-4a76-88fe-40f0f7b9e97f" containerName="payload-pre-processing" containerID="cri-o://ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98" gracePeriod=30 Sep 10 15:09:25.198019 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.197995 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:09:25.232288 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.232263 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:09:25.233683 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.233659 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-metrics-certs\") pod \"c911aae3-876b-4ab6-96bb-f72a95936a5d\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " Sep 10 15:09:25.233757 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.233733 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwr5q\" (UniqueName: \"kubernetes.io/projected/c911aae3-876b-4ab6-96bb-f72a95936a5d-kube-api-access-lwr5q\") pod \"c911aae3-876b-4ab6-96bb-f72a95936a5d\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " Sep 10 15:09:25.233824 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.233801 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c911aae3-876b-4ab6-96bb-f72a95936a5d-openshift-service-ca\") pod \"c911aae3-876b-4ab6-96bb-f72a95936a5d\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " Sep 10 15:09:25.233931 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.233915 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls\") pod \"c911aae3-876b-4ab6-96bb-f72a95936a5d\" (UID: \"c911aae3-876b-4ab6-96bb-f72a95936a5d\") " Sep 10 15:09:25.234169 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.234131 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c911aae3-876b-4ab6-96bb-f72a95936a5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c911aae3-876b-4ab6-96bb-f72a95936a5d" (UID: "c911aae3-876b-4ab6-96bb-f72a95936a5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:09:25.234328 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.234307 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c911aae3-876b-4ab6-96bb-f72a95936a5d-openshift-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:25.235823 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.235795 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c911aae3-876b-4ab6-96bb-f72a95936a5d" (UID: "c911aae3-876b-4ab6-96bb-f72a95936a5d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:09:25.235913 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.235817 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c911aae3-876b-4ab6-96bb-f72a95936a5d-kube-api-access-lwr5q" (OuterVolumeSpecName: "kube-api-access-lwr5q") pod "c911aae3-876b-4ab6-96bb-f72a95936a5d" (UID: "c911aae3-876b-4ab6-96bb-f72a95936a5d"). InnerVolumeSpecName "kube-api-access-lwr5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:09:25.236318 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.236301 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c911aae3-876b-4ab6-96bb-f72a95936a5d" (UID: "c911aae3-876b-4ab6-96bb-f72a95936a5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:09:25.337017 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.336877 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szwn4\" (UniqueName: \"kubernetes.io/projected/69349758-fe24-4a76-88fe-40f0f7b9e97f-kube-api-access-szwn4\") pod \"69349758-fe24-4a76-88fe-40f0f7b9e97f\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " Sep 10 15:09:25.337017 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.336930 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69349758-fe24-4a76-88fe-40f0f7b9e97f-plugins-config-volume\") pod \"69349758-fe24-4a76-88fe-40f0f7b9e97f\" (UID: \"69349758-fe24-4a76-88fe-40f0f7b9e97f\") " Sep 10 15:09:25.337501 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.337467 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwr5q\" (UniqueName: \"kubernetes.io/projected/c911aae3-876b-4ab6-96bb-f72a95936a5d-kube-api-access-lwr5q\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:25.337501 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.337492 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-maas-api-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:25.337501 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.337504 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c911aae3-876b-4ab6-96bb-f72a95936a5d-metrics-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:25.337721 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.337543 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69349758-fe24-4a76-88fe-40f0f7b9e97f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69349758-fe24-4a76-88fe-40f0f7b9e97f" (UID: "69349758-fe24-4a76-88fe-40f0f7b9e97f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:09:25.339354 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.339319 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69349758-fe24-4a76-88fe-40f0f7b9e97f-kube-api-access-szwn4" (OuterVolumeSpecName: "kube-api-access-szwn4") pod "69349758-fe24-4a76-88fe-40f0f7b9e97f" (UID: "69349758-fe24-4a76-88fe-40f0f7b9e97f"). InnerVolumeSpecName "kube-api-access-szwn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:09:25.438356 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.438281 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szwn4\" (UniqueName: \"kubernetes.io/projected/69349758-fe24-4a76-88fe-40f0f7b9e97f-kube-api-access-szwn4\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:25.438356 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.438306 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69349758-fe24-4a76-88fe-40f0f7b9e97f-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:25.662118 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.662077 2584 generic.go:356] "Generic (PLEG): container finished" podID="69349758-fe24-4a76-88fe-40f0f7b9e97f" containerID="ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98" exitCode=0 Sep 10 15:09:25.662336 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.662137 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" Sep 10 15:09:25.662336 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.662161 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" event={"ID":"69349758-fe24-4a76-88fe-40f0f7b9e97f","Type":"ContainerDied","Data":"ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98"} Sep 10 15:09:25.662336 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.662264 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6" event={"ID":"69349758-fe24-4a76-88fe-40f0f7b9e97f","Type":"ContainerDied","Data":"d4ec4c91c185cd969c23f52c9196ad1880dfa2296b6ad07faf015f08975299e4"} Sep 10 15:09:25.662336 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.662292 2584 scope.go:117] "RemoveContainer" containerID="ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98" Sep 10 15:09:25.663694 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.663664 2584 generic.go:356] "Generic (PLEG): container finished" podID="c911aae3-876b-4ab6-96bb-f72a95936a5d" containerID="6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8" exitCode=0 Sep 10 15:09:25.663811 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.663711 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" Sep 10 15:09:25.663811 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.663723 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" event={"ID":"c911aae3-876b-4ab6-96bb-f72a95936a5d","Type":"ContainerDied","Data":"6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8"} Sep 10 15:09:25.663811 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.663772 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd" event={"ID":"c911aae3-876b-4ab6-96bb-f72a95936a5d","Type":"ContainerDied","Data":"adf06916d73bc5bf938198df8df2c42c20a6639405e18acc66c6e863c744392d"} Sep 10 15:09:25.671917 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.671774 2584 scope.go:117] "RemoveContainer" containerID="ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98" Sep 10 15:09:25.672091 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:09:25.672066 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98\": container with ID starting with ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98 not found: ID does not exist" containerID="ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98" Sep 10 15:09:25.672149 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.672103 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98"} err="failed to get container status \"ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98\": rpc error: code = NotFound desc = could not find container \"ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98\": container with ID starting with ada8aa51f1f7ba12ae73cdc38afb55a76be48f1fff172be719a0d14b00646e98 not found: ID does not exist" Sep 10 15:09:25.672149 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.672127 2584 scope.go:117] "RemoveContainer" containerID="6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8" Sep 10 15:09:25.680032 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.680018 2584 scope.go:117] "RemoveContainer" containerID="6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8" Sep 10 15:09:25.680310 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:09:25.680294 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8\": container with ID starting with 6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8 not found: ID does not exist" containerID="6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8" Sep 10 15:09:25.680376 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.680315 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8"} err="failed to get container status \"6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8\": rpc error: code = NotFound desc = could not find container \"6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8\": container with ID starting with 6fa02c6b92c72e0769a6f50f1460e1f3493330a6be6406a73316094e4e48b6e8 not found: ID does not exist" Sep 10 15:09:25.688766 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.688717 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6"] Sep 10 15:09:25.693071 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.693047 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-799b5a-677d8498db-km7q6"] Sep 10 15:09:25.704967 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.704949 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd"] Sep 10 15:09:25.709730 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.709710 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-799b5a-6f4d66988-dm4wd"] Sep 10 15:09:25.764676 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.764644 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69349758-fe24-4a76-88fe-40f0f7b9e97f" path="/var/lib/kubelet/pods/69349758-fe24-4a76-88fe-40f0f7b9e97f/volumes" Sep 10 15:09:25.765041 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:25.765028 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c911aae3-876b-4ab6-96bb-f72a95936a5d" path="/var/lib/kubelet/pods/c911aae3-876b-4ab6-96bb-f72a95936a5d/volumes" Sep 10 15:09:51.259482 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.259447 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl"] Sep 10 15:09:51.260154 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.260088 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" podUID="81233af0-d7e0-40ae-8800-fcbdb159e949" containerName="maas-api" containerID="cri-o://8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f" gracePeriod=30 Sep 10 15:09:51.511980 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.511915 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:09:51.681482 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.681449 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls\") pod \"81233af0-d7e0-40ae-8800-fcbdb159e949\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " Sep 10 15:09:51.681674 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.681523 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khl4c\" (UniqueName: \"kubernetes.io/projected/81233af0-d7e0-40ae-8800-fcbdb159e949-kube-api-access-khl4c\") pod \"81233af0-d7e0-40ae-8800-fcbdb159e949\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " Sep 10 15:09:51.681674 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.681566 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-metrics-certs\") pod \"81233af0-d7e0-40ae-8800-fcbdb159e949\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " Sep 10 15:09:51.681674 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.681589 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81233af0-d7e0-40ae-8800-fcbdb159e949-openshift-service-ca\") pod \"81233af0-d7e0-40ae-8800-fcbdb159e949\" (UID: \"81233af0-d7e0-40ae-8800-fcbdb159e949\") " Sep 10 15:09:51.682003 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.681966 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81233af0-d7e0-40ae-8800-fcbdb159e949-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81233af0-d7e0-40ae-8800-fcbdb159e949" (UID: "81233af0-d7e0-40ae-8800-fcbdb159e949"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:09:51.683753 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.683725 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81233af0-d7e0-40ae-8800-fcbdb159e949" (UID: "81233af0-d7e0-40ae-8800-fcbdb159e949"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:09:51.684084 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.684061 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "81233af0-d7e0-40ae-8800-fcbdb159e949" (UID: "81233af0-d7e0-40ae-8800-fcbdb159e949"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:09:51.684084 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.684073 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81233af0-d7e0-40ae-8800-fcbdb159e949-kube-api-access-khl4c" (OuterVolumeSpecName: "kube-api-access-khl4c") pod "81233af0-d7e0-40ae-8800-fcbdb159e949" (UID: "81233af0-d7e0-40ae-8800-fcbdb159e949"). InnerVolumeSpecName "kube-api-access-khl4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:09:51.782721 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.782696 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-maas-api-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:51.782721 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.782718 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khl4c\" (UniqueName: \"kubernetes.io/projected/81233af0-d7e0-40ae-8800-fcbdb159e949-kube-api-access-khl4c\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:51.782875 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.782728 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81233af0-d7e0-40ae-8800-fcbdb159e949-metrics-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:51.782875 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.782738 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81233af0-d7e0-40ae-8800-fcbdb159e949-openshift-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:09:51.791722 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.791690 2584 generic.go:356] "Generic (PLEG): container finished" podID="81233af0-d7e0-40ae-8800-fcbdb159e949" containerID="8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f" exitCode=0 Sep 10 15:09:51.791821 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.791735 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" Sep 10 15:09:51.791821 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.791772 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" event={"ID":"81233af0-d7e0-40ae-8800-fcbdb159e949","Type":"ContainerDied","Data":"8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f"} Sep 10 15:09:51.791919 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.791830 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl" event={"ID":"81233af0-d7e0-40ae-8800-fcbdb159e949","Type":"ContainerDied","Data":"7c848e620cdb6b85351a343ea503ba419562cf4513b6fef47934e24feeac3885"} Sep 10 15:09:51.791919 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.791854 2584 scope.go:117] "RemoveContainer" containerID="8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f" Sep 10 15:09:51.800836 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.800804 2584 scope.go:117] "RemoveContainer" containerID="8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f" Sep 10 15:09:51.801091 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:09:51.801065 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f\": container with ID starting with 8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f not found: ID does not exist" containerID="8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f" Sep 10 15:09:51.801143 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.801092 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f"} err="failed to get container status \"8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f\": rpc error: code = NotFound desc = could not find container \"8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f\": container with ID starting with 8ebdf5538d6856c87074f451c78f2381a81dd3f6f28da97ad84c6fb19f4f414f not found: ID does not exist" Sep 10 15:09:51.810445 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.810417 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl"] Sep 10 15:09:51.814552 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:51.814525 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-b022ee-6c5b5846d4-jkskl"] Sep 10 15:09:53.763079 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:09:53.763044 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81233af0-d7e0-40ae-8800-fcbdb159e949" path="/var/lib/kubelet/pods/81233af0-d7e0-40ae-8800-fcbdb159e949/volumes" Sep 10 15:10:40.788766 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.788731 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd"] Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789096 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81233af0-d7e0-40ae-8800-fcbdb159e949" containerName="maas-api" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789106 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="81233af0-d7e0-40ae-8800-fcbdb159e949" containerName="maas-api" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789120 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c911aae3-876b-4ab6-96bb-f72a95936a5d" containerName="maas-api" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789125 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c911aae3-876b-4ab6-96bb-f72a95936a5d" containerName="maas-api" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789140 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69349758-fe24-4a76-88fe-40f0f7b9e97f" containerName="payload-pre-processing" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789145 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="69349758-fe24-4a76-88fe-40f0f7b9e97f" containerName="payload-pre-processing" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789238 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="69349758-fe24-4a76-88fe-40f0f7b9e97f" containerName="payload-pre-processing" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789248 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c911aae3-876b-4ab6-96bb-f72a95936a5d" containerName="maas-api" Sep 10 15:10:40.789282 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.789256 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="81233af0-d7e0-40ae-8800-fcbdb159e949" containerName="maas-api" Sep 10 15:10:40.791980 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.791964 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" Sep 10 15:10:40.795156 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.795136 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:10:40.795277 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.795153 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\"" Sep 10 15:10:40.801809 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.801782 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd"] Sep 10 15:10:40.924121 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:40.924082 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f82930-d43f-4e52-b78c-76b44f0993a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd\" (UID: \"e4f82930-d43f-4e52-b78c-76b44f0993a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" Sep 10 15:10:41.025213 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:41.025165 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f82930-d43f-4e52-b78c-76b44f0993a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd\" (UID: \"e4f82930-d43f-4e52-b78c-76b44f0993a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" Sep 10 15:10:41.025810 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:41.025788 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f82930-d43f-4e52-b78c-76b44f0993a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd\" (UID: \"e4f82930-d43f-4e52-b78c-76b44f0993a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" Sep 10 15:10:41.103337 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:41.103261 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" Sep 10 15:10:41.378123 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:41.378100 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd"] Sep 10 15:10:41.380298 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:10:41.380270 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f82930_d43f_4e52_b78c_76b44f0993a8.slice/crio-6826e57babbbd34659f3dab4d6316ef00a3c4375634ae183853937f87d0c2563 WatchSource:0}: Error finding container 6826e57babbbd34659f3dab4d6316ef00a3c4375634ae183853937f87d0c2563: Status 404 returned error can't find the container with id 6826e57babbbd34659f3dab4d6316ef00a3c4375634ae183853937f87d0c2563 Sep 10 15:10:42.042478 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:42.042435 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" event={"ID":"e4f82930-d43f-4e52-b78c-76b44f0993a8","Type":"ContainerStarted","Data":"f867a06e0447b7b1cc57f18a5f1b6a4a803dd1cb266dd78e13ad3ea6876aab78"} Sep 10 15:10:42.042863 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:42.042530 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" event={"ID":"e4f82930-d43f-4e52-b78c-76b44f0993a8","Type":"ContainerStarted","Data":"6826e57babbbd34659f3dab4d6316ef00a3c4375634ae183853937f87d0c2563"} Sep 10 15:10:42.059238 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:42.059178 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" podStartSLOduration=2.059163794 podStartE2EDuration="2.059163794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:10:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:10:42.058619707 +0000 UTC m=+1698.730645480" watchObservedRunningTime="2026-09-10 15:10:42.059163794 +0000 UTC m=+1698.731189565" Sep 10 15:10:43.048732 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:43.048687 2584 generic.go:356] "Generic (PLEG): container finished" podID="e4f82930-d43f-4e52-b78c-76b44f0993a8" containerID="f867a06e0447b7b1cc57f18a5f1b6a4a803dd1cb266dd78e13ad3ea6876aab78" exitCode=0 Sep 10 15:10:43.049106 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:43.048753 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" event={"ID":"e4f82930-d43f-4e52-b78c-76b44f0993a8","Type":"ContainerDied","Data":"f867a06e0447b7b1cc57f18a5f1b6a4a803dd1cb266dd78e13ad3ea6876aab78"} Sep 10 15:10:44.191432 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:44.191406 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" Sep 10 15:10:44.356395 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:44.356309 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f82930-d43f-4e52-b78c-76b44f0993a8-maas-api-service-ca\") pod \"e4f82930-d43f-4e52-b78c-76b44f0993a8\" (UID: \"e4f82930-d43f-4e52-b78c-76b44f0993a8\") " Sep 10 15:10:44.356652 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:44.356630 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f82930-d43f-4e52-b78c-76b44f0993a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4f82930-d43f-4e52-b78c-76b44f0993a8" (UID: "e4f82930-d43f-4e52-b78c-76b44f0993a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:10:44.457672 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:44.457644 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f82930-d43f-4e52-b78c-76b44f0993a8-maas-api-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:10:45.059443 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:45.059407 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" event={"ID":"e4f82930-d43f-4e52-b78c-76b44f0993a8","Type":"ContainerDied","Data":"6826e57babbbd34659f3dab4d6316ef00a3c4375634ae183853937f87d0c2563"} Sep 10 15:10:45.059639 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:45.059459 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6826e57babbbd34659f3dab4d6316ef00a3c4375634ae183853937f87d0c2563" Sep 10 15:10:45.059639 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:45.059414 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd" Sep 10 15:10:51.345303 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.345217 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67"] Sep 10 15:10:51.345746 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.345471 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" podUID="02548f5b-563d-4965-964f-658db34581ce" containerName="payload-processing" containerID="cri-o://a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545" gracePeriod=30 Sep 10 15:10:51.566891 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.566859 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd"] Sep 10 15:10:51.569762 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.569739 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cb5c8c-7zkfd"] Sep 10 15:10:51.604937 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.604875 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:10:51.622905 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.622873 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68b6m\" (UniqueName: \"kubernetes.io/projected/02548f5b-563d-4965-964f-658db34581ce-kube-api-access-68b6m\") pod \"02548f5b-563d-4965-964f-658db34581ce\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " Sep 10 15:10:51.623072 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.622921 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02548f5b-563d-4965-964f-658db34581ce-plugins-config-volume\") pod \"02548f5b-563d-4965-964f-658db34581ce\" (UID: \"02548f5b-563d-4965-964f-658db34581ce\") " Sep 10 15:10:51.623867 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.623421 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02548f5b-563d-4965-964f-658db34581ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02548f5b-563d-4965-964f-658db34581ce" (UID: "02548f5b-563d-4965-964f-658db34581ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:10:51.623867 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.623800 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02548f5b-563d-4965-964f-658db34581ce-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:10:51.628410 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.627515 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02548f5b-563d-4965-964f-658db34581ce-kube-api-access-68b6m" (OuterVolumeSpecName: "kube-api-access-68b6m") pod "02548f5b-563d-4965-964f-658db34581ce" (UID: "02548f5b-563d-4965-964f-658db34581ce"). InnerVolumeSpecName "kube-api-access-68b6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:10:51.724674 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.724637 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68b6m\" (UniqueName: \"kubernetes.io/projected/02548f5b-563d-4965-964f-658db34581ce-kube-api-access-68b6m\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:10:51.762433 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:51.762400 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4f82930-d43f-4e52-b78c-76b44f0993a8" path="/var/lib/kubelet/pods/e4f82930-d43f-4e52-b78c-76b44f0993a8/volumes" Sep 10 15:10:52.094374 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.094342 2584 generic.go:356] "Generic (PLEG): container finished" podID="02548f5b-563d-4965-964f-658db34581ce" containerID="a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545" exitCode=0 Sep 10 15:10:52.094554 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.094388 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" Sep 10 15:10:52.094554 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.094438 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" event={"ID":"02548f5b-563d-4965-964f-658db34581ce","Type":"ContainerDied","Data":"a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545"} Sep 10 15:10:52.094554 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.094494 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67" event={"ID":"02548f5b-563d-4965-964f-658db34581ce","Type":"ContainerDied","Data":"5c48160218685ba2aff6a5fcd3df88e539eb4c4d05f8c42e3ceec493b554438d"} Sep 10 15:10:52.094554 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.094522 2584 scope.go:117] "RemoveContainer" containerID="a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545" Sep 10 15:10:52.103119 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.103101 2584 scope.go:117] "RemoveContainer" containerID="a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545" Sep 10 15:10:52.103398 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:10:52.103372 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545\": container with ID starting with a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545 not found: ID does not exist" containerID="a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545" Sep 10 15:10:52.103507 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.103399 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545"} err="failed to get container status \"a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545\": rpc error: code = NotFound desc = could not find container \"a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545\": container with ID starting with a525dcdd758dd3d4975ad52a594ffddaf272bcff7c8db0238fb60032f870d545 not found: ID does not exist" Sep 10 15:10:52.111910 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.111887 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67"] Sep 10 15:10:52.115979 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:52.115959 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-cb5c8c-5cf6c4bbb7-lfd67"] Sep 10 15:10:53.762600 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:53.762569 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02548f5b-563d-4965-964f-658db34581ce" path="/var/lib/kubelet/pods/02548f5b-563d-4965-964f-658db34581ce/volumes" Sep 10 15:10:58.400717 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.400685 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk"] Sep 10 15:10:58.401130 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.401054 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02548f5b-563d-4965-964f-658db34581ce" containerName="payload-processing" Sep 10 15:10:58.401130 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.401066 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="02548f5b-563d-4965-964f-658db34581ce" containerName="payload-processing" Sep 10 15:10:58.401130 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.401076 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f82930-d43f-4e52-b78c-76b44f0993a8" containerName="revoke-keys" Sep 10 15:10:58.401130 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.401083 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f82930-d43f-4e52-b78c-76b44f0993a8" containerName="revoke-keys" Sep 10 15:10:58.401275 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.401147 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f82930-d43f-4e52-b78c-76b44f0993a8" containerName="revoke-keys" Sep 10 15:10:58.401275 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.401158 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="02548f5b-563d-4965-964f-658db34581ce" containerName="payload-processing" Sep 10 15:10:58.403911 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.403894 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" Sep 10 15:10:58.407071 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.407044 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:10:58.407219 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.407076 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\"" Sep 10 15:10:58.414159 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.414136 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk"] Sep 10 15:10:58.484275 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.484235 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf247df-8e07-46c1-828f-8b8a2bbb3be8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk\" (UID: \"3cf247df-8e07-46c1-828f-8b8a2bbb3be8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" Sep 10 15:10:58.585892 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.585859 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf247df-8e07-46c1-828f-8b8a2bbb3be8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk\" (UID: \"3cf247df-8e07-46c1-828f-8b8a2bbb3be8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" Sep 10 15:10:58.586534 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.586513 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf247df-8e07-46c1-828f-8b8a2bbb3be8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk\" (UID: \"3cf247df-8e07-46c1-828f-8b8a2bbb3be8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" Sep 10 15:10:58.715295 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.715267 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" Sep 10 15:10:58.990818 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:58.990731 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk"] Sep 10 15:10:58.994083 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:10:58.994056 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf247df_8e07_46c1_828f_8b8a2bbb3be8.slice/crio-be9834dc565c879665d0ccdc03d432d45b507123c54c0ea471c40d5bfbd8ece9 WatchSource:0}: Error finding container be9834dc565c879665d0ccdc03d432d45b507123c54c0ea471c40d5bfbd8ece9: Status 404 returned error can't find the container with id be9834dc565c879665d0ccdc03d432d45b507123c54c0ea471c40d5bfbd8ece9 Sep 10 15:10:59.132516 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:10:59.132477 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" event={"ID":"3cf247df-8e07-46c1-828f-8b8a2bbb3be8","Type":"ContainerStarted","Data":"be9834dc565c879665d0ccdc03d432d45b507123c54c0ea471c40d5bfbd8ece9"} Sep 10 15:11:00.139157 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:00.139119 2584 generic.go:356] "Generic (PLEG): container finished" podID="3cf247df-8e07-46c1-828f-8b8a2bbb3be8" containerID="e484d7ad51b3427533ad1ddfbcc2817574d535ad12afcc570401a13150a04e5b" exitCode=0 Sep 10 15:11:00.139634 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:00.139175 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" event={"ID":"3cf247df-8e07-46c1-828f-8b8a2bbb3be8","Type":"ContainerDied","Data":"e484d7ad51b3427533ad1ddfbcc2817574d535ad12afcc570401a13150a04e5b"} Sep 10 15:11:01.278299 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:01.278275 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" Sep 10 15:11:01.309255 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:01.309226 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf247df-8e07-46c1-828f-8b8a2bbb3be8-maas-api-service-ca\") pod \"3cf247df-8e07-46c1-828f-8b8a2bbb3be8\" (UID: \"3cf247df-8e07-46c1-828f-8b8a2bbb3be8\") " Sep 10 15:11:01.309573 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:01.309550 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf247df-8e07-46c1-828f-8b8a2bbb3be8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3cf247df-8e07-46c1-828f-8b8a2bbb3be8" (UID: "3cf247df-8e07-46c1-828f-8b8a2bbb3be8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:11:01.410541 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:01.410459 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf247df-8e07-46c1-828f-8b8a2bbb3be8-maas-api-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:02.152956 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:02.152873 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" Sep 10 15:11:02.152956 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:02.152908 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk" event={"ID":"3cf247df-8e07-46c1-828f-8b8a2bbb3be8","Type":"ContainerDied","Data":"be9834dc565c879665d0ccdc03d432d45b507123c54c0ea471c40d5bfbd8ece9"} Sep 10 15:11:02.152956 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:02.152950 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be9834dc565c879665d0ccdc03d432d45b507123c54c0ea471c40d5bfbd8ece9" Sep 10 15:11:08.997506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:08.997443 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p"] Sep 10 15:11:09.001643 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.001613 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-16ad45-29817540-frz5p"] Sep 10 15:11:09.030153 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.030111 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95"] Sep 10 15:11:09.030462 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.030407 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" podUID="dd1fbb42-ad63-4be4-a87f-70dd056c02fd" containerName="payload-processing" containerID="cri-o://e37ee08947476de2a059dd72a49840d0c92bf70d807b460214c67c082546d283" gracePeriod=30 Sep 10 15:11:09.039628 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.038994 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6"] Sep 10 15:11:09.039628 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.039257 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" podUID="478f8ae7-0112-4457-9b50-22e311734e36" containerName="payload-pre-processing" containerID="cri-o://452f6928acd975782d417ff45ef9dd7d1be57322a0a02591fef0818ae6f068d1" gracePeriod=30 Sep 10 15:11:09.193439 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.193402 2584 generic.go:356] "Generic (PLEG): container finished" podID="478f8ae7-0112-4457-9b50-22e311734e36" containerID="452f6928acd975782d417ff45ef9dd7d1be57322a0a02591fef0818ae6f068d1" exitCode=0 Sep 10 15:11:09.193636 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.193509 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" event={"ID":"478f8ae7-0112-4457-9b50-22e311734e36","Type":"ContainerDied","Data":"452f6928acd975782d417ff45ef9dd7d1be57322a0a02591fef0818ae6f068d1"} Sep 10 15:11:09.195703 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.195669 2584 generic.go:356] "Generic (PLEG): container finished" podID="dd1fbb42-ad63-4be4-a87f-70dd056c02fd" containerID="e37ee08947476de2a059dd72a49840d0c92bf70d807b460214c67c082546d283" exitCode=0 Sep 10 15:11:09.195820 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.195741 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" event={"ID":"dd1fbb42-ad63-4be4-a87f-70dd056c02fd","Type":"ContainerDied","Data":"e37ee08947476de2a059dd72a49840d0c92bf70d807b460214c67c082546d283"} Sep 10 15:11:09.226447 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.226401 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk"] Sep 10 15:11:09.229364 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.229330 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-16ad45-b2bgk"] Sep 10 15:11:09.297688 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.297656 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 15:11:09.328327 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.328296 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 15:11:09.378987 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.378948 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj77w\" (UniqueName: \"kubernetes.io/projected/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-kube-api-access-cj77w\") pod \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " Sep 10 15:11:09.379371 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.379056 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nww7b\" (UniqueName: \"kubernetes.io/projected/478f8ae7-0112-4457-9b50-22e311734e36-kube-api-access-nww7b\") pod \"478f8ae7-0112-4457-9b50-22e311734e36\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " Sep 10 15:11:09.379371 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.379102 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478f8ae7-0112-4457-9b50-22e311734e36-plugins-config-volume\") pod \"478f8ae7-0112-4457-9b50-22e311734e36\" (UID: \"478f8ae7-0112-4457-9b50-22e311734e36\") " Sep 10 15:11:09.379371 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.379148 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-plugins-config-volume\") pod \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\" (UID: \"dd1fbb42-ad63-4be4-a87f-70dd056c02fd\") " Sep 10 15:11:09.379813 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.379768 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd1fbb42-ad63-4be4-a87f-70dd056c02fd" (UID: "dd1fbb42-ad63-4be4-a87f-70dd056c02fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:11:09.380186 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.380157 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/478f8ae7-0112-4457-9b50-22e311734e36-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "478f8ae7-0112-4457-9b50-22e311734e36" (UID: "478f8ae7-0112-4457-9b50-22e311734e36"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:11:09.381702 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.381676 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478f8ae7-0112-4457-9b50-22e311734e36-kube-api-access-nww7b" (OuterVolumeSpecName: "kube-api-access-nww7b") pod "478f8ae7-0112-4457-9b50-22e311734e36" (UID: "478f8ae7-0112-4457-9b50-22e311734e36"). InnerVolumeSpecName "kube-api-access-nww7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:11:09.381794 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.381746 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-kube-api-access-cj77w" (OuterVolumeSpecName: "kube-api-access-cj77w") pod "dd1fbb42-ad63-4be4-a87f-70dd056c02fd" (UID: "dd1fbb42-ad63-4be4-a87f-70dd056c02fd"). InnerVolumeSpecName "kube-api-access-cj77w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:11:09.480796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.480729 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478f8ae7-0112-4457-9b50-22e311734e36-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:09.480796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.480781 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:09.480796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.480797 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj77w\" (UniqueName: \"kubernetes.io/projected/dd1fbb42-ad63-4be4-a87f-70dd056c02fd-kube-api-access-cj77w\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:09.480796 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.480809 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nww7b\" (UniqueName: \"kubernetes.io/projected/478f8ae7-0112-4457-9b50-22e311734e36-kube-api-access-nww7b\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:09.763571 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.763537 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf247df-8e07-46c1-828f-8b8a2bbb3be8" path="/var/lib/kubelet/pods/3cf247df-8e07-46c1-828f-8b8a2bbb3be8/volumes" Sep 10 15:11:09.763855 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:09.763842 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5128c045-4d08-4090-afde-7ee26439cf75" path="/var/lib/kubelet/pods/5128c045-4d08-4090-afde-7ee26439cf75/volumes" Sep 10 15:11:10.201832 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.201799 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" Sep 10 15:11:10.202303 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.201792 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6" event={"ID":"478f8ae7-0112-4457-9b50-22e311734e36","Type":"ContainerDied","Data":"b4abb2566da1004d2d54941ce657f0ba2c05d9741e50da31f320a6af98c3f97d"} Sep 10 15:11:10.202303 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.201995 2584 scope.go:117] "RemoveContainer" containerID="452f6928acd975782d417ff45ef9dd7d1be57322a0a02591fef0818ae6f068d1" Sep 10 15:11:10.203450 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.203248 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" event={"ID":"dd1fbb42-ad63-4be4-a87f-70dd056c02fd","Type":"ContainerDied","Data":"ea49e0edd9f08708a12614c2a9c318e5d8abd9513671324f510488dba4c7f8c2"} Sep 10 15:11:10.203450 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.203281 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95" Sep 10 15:11:10.211066 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.211045 2584 scope.go:117] "RemoveContainer" containerID="e37ee08947476de2a059dd72a49840d0c92bf70d807b460214c67c082546d283" Sep 10 15:11:10.221492 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.221459 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6"] Sep 10 15:11:10.226053 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.226025 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-16ad45-66b475df66-sccj6"] Sep 10 15:11:10.240963 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.240925 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95"] Sep 10 15:11:10.248960 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:10.248928 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-16ad45-6dcc655c6f-zhd95"] Sep 10 15:11:11.764868 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:11.764813 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="478f8ae7-0112-4457-9b50-22e311734e36" path="/var/lib/kubelet/pods/478f8ae7-0112-4457-9b50-22e311734e36/volumes" Sep 10 15:11:11.765482 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:11.765352 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd1fbb42-ad63-4be4-a87f-70dd056c02fd" path="/var/lib/kubelet/pods/dd1fbb42-ad63-4be4-a87f-70dd056c02fd/volumes" Sep 10 15:11:16.277552 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.277516 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr"] Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.277898 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd1fbb42-ad63-4be4-a87f-70dd056c02fd" containerName="payload-processing" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.277910 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd1fbb42-ad63-4be4-a87f-70dd056c02fd" containerName="payload-processing" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.277929 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf247df-8e07-46c1-828f-8b8a2bbb3be8" containerName="revoke-keys" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.277935 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf247df-8e07-46c1-828f-8b8a2bbb3be8" containerName="revoke-keys" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.277949 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="478f8ae7-0112-4457-9b50-22e311734e36" containerName="payload-pre-processing" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.277956 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="478f8ae7-0112-4457-9b50-22e311734e36" containerName="payload-pre-processing" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.278011 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="478f8ae7-0112-4457-9b50-22e311734e36" containerName="payload-pre-processing" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.278024 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf247df-8e07-46c1-828f-8b8a2bbb3be8" containerName="revoke-keys" Sep 10 15:11:16.278037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.278030 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd1fbb42-ad63-4be4-a87f-70dd056c02fd" containerName="payload-processing" Sep 10 15:11:16.282506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.282485 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" Sep 10 15:11:16.285495 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.285470 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:11:16.285635 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.285553 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pc7s\"" Sep 10 15:11:16.291967 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.291939 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr"] Sep 10 15:11:16.352162 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.352120 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c7a886-c156-465b-8294-d293a538a349-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr\" (UID: \"24c7a886-c156-465b-8294-d293a538a349\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" Sep 10 15:11:16.453324 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.453283 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c7a886-c156-465b-8294-d293a538a349-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr\" (UID: \"24c7a886-c156-465b-8294-d293a538a349\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" Sep 10 15:11:16.453981 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.453956 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c7a886-c156-465b-8294-d293a538a349-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr\" (UID: \"24c7a886-c156-465b-8294-d293a538a349\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" Sep 10 15:11:16.595100 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.595004 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" Sep 10 15:11:16.869896 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:16.869870 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr"] Sep 10 15:11:16.872522 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:11:16.872484 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c7a886_c156_465b_8294_d293a538a349.slice/crio-0309e25512c3693a4a4376cda65274f471223540f6f1eff2f025cde1d7816fe8 WatchSource:0}: Error finding container 0309e25512c3693a4a4376cda65274f471223540f6f1eff2f025cde1d7816fe8: Status 404 returned error can't find the container with id 0309e25512c3693a4a4376cda65274f471223540f6f1eff2f025cde1d7816fe8 Sep 10 15:11:17.244071 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:17.244029 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" event={"ID":"24c7a886-c156-465b-8294-d293a538a349","Type":"ContainerStarted","Data":"87fc6b4ca8a754683a0d54b80aeb8108e0500b791bdf1d9387f7d9069c32514f"} Sep 10 15:11:17.244301 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:17.244090 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" event={"ID":"24c7a886-c156-465b-8294-d293a538a349","Type":"ContainerStarted","Data":"0309e25512c3693a4a4376cda65274f471223540f6f1eff2f025cde1d7816fe8"} Sep 10 15:11:17.263125 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:17.263080 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" podStartSLOduration=1.263064889 podStartE2EDuration="1.263064889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:11:17.261750339 +0000 UTC m=+1733.933776111" watchObservedRunningTime="2026-09-10 15:11:17.263064889 +0000 UTC m=+1733.935090661" Sep 10 15:11:18.250350 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:18.250315 2584 generic.go:356] "Generic (PLEG): container finished" podID="24c7a886-c156-465b-8294-d293a538a349" containerID="87fc6b4ca8a754683a0d54b80aeb8108e0500b791bdf1d9387f7d9069c32514f" exitCode=0 Sep 10 15:11:18.250743 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:18.250396 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" event={"ID":"24c7a886-c156-465b-8294-d293a538a349","Type":"ContainerDied","Data":"87fc6b4ca8a754683a0d54b80aeb8108e0500b791bdf1d9387f7d9069c32514f"} Sep 10 15:11:19.391497 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:19.391471 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" Sep 10 15:11:19.485243 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:19.485189 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c7a886-c156-465b-8294-d293a538a349-maas-api-service-ca\") pod \"24c7a886-c156-465b-8294-d293a538a349\" (UID: \"24c7a886-c156-465b-8294-d293a538a349\") " Sep 10 15:11:19.485549 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:19.485529 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c7a886-c156-465b-8294-d293a538a349-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24c7a886-c156-465b-8294-d293a538a349" (UID: "24c7a886-c156-465b-8294-d293a538a349"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:11:19.586681 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:19.586596 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c7a886-c156-465b-8294-d293a538a349-maas-api-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:20.261214 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:20.261167 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" event={"ID":"24c7a886-c156-465b-8294-d293a538a349","Type":"ContainerDied","Data":"0309e25512c3693a4a4376cda65274f471223540f6f1eff2f025cde1d7816fe8"} Sep 10 15:11:20.261214 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:20.261189 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr" Sep 10 15:11:20.261435 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:20.261232 2584 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0309e25512c3693a4a4376cda65274f471223540f6f1eff2f025cde1d7816fe8" Sep 10 15:11:23.951713 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:23.951679 2584 scope.go:117] "RemoveContainer" containerID="380f7390d59109a3a696ffaeaddd9fedae533dd21388f1f795676739cc691218" Sep 10 15:11:23.960004 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:23.959981 2584 scope.go:117] "RemoveContainer" containerID="f18052aef2d64ebd1e408183ee834d4cf5fff8d619e47b8205ea53867e50295f" Sep 10 15:11:23.967310 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:23.967290 2584 scope.go:117] "RemoveContainer" containerID="2f8bda4ce86dab0719b09b947eabdc80321b86aeefb1a9ced0c1fc754bcffaa3" Sep 10 15:11:26.859426 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:26.859388 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg"] Sep 10 15:11:26.859906 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:26.859639 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" podUID="b1107f52-46b9-40cb-9865-1f5aa38e6717" containerName="payload-processing" containerID="cri-o://8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd" gracePeriod=30 Sep 10 15:11:26.881168 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:26.881141 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h"] Sep 10 15:11:26.881394 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:26.881349 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" podUID="b58aec8d-ec8e-4238-9e8f-d35a722b3b39" containerName="payload-pre-processing" containerID="cri-o://4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a" gracePeriod=30 Sep 10 15:11:27.152671 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.152640 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:11:27.155538 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.155512 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:11:27.159571 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.159548 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr"] Sep 10 15:11:27.163664 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.163644 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-3f81b7-q5xvr"] Sep 10 15:11:27.252305 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.252274 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6msss\" (UniqueName: \"kubernetes.io/projected/b1107f52-46b9-40cb-9865-1f5aa38e6717-kube-api-access-6msss\") pod \"b1107f52-46b9-40cb-9865-1f5aa38e6717\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " Sep 10 15:11:27.252506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.252344 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8wf4\" (UniqueName: \"kubernetes.io/projected/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-kube-api-access-d8wf4\") pod \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " Sep 10 15:11:27.252506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.252399 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1107f52-46b9-40cb-9865-1f5aa38e6717-plugins-config-volume\") pod \"b1107f52-46b9-40cb-9865-1f5aa38e6717\" (UID: \"b1107f52-46b9-40cb-9865-1f5aa38e6717\") " Sep 10 15:11:27.252506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.252438 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-plugins-config-volume\") pod \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\" (UID: \"b58aec8d-ec8e-4238-9e8f-d35a722b3b39\") " Sep 10 15:11:27.252812 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.252788 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1107f52-46b9-40cb-9865-1f5aa38e6717-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1107f52-46b9-40cb-9865-1f5aa38e6717" (UID: "b1107f52-46b9-40cb-9865-1f5aa38e6717"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:11:27.252886 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.252835 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b58aec8d-ec8e-4238-9e8f-d35a722b3b39" (UID: "b58aec8d-ec8e-4238-9e8f-d35a722b3b39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:11:27.254519 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.254498 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-kube-api-access-d8wf4" (OuterVolumeSpecName: "kube-api-access-d8wf4") pod "b58aec8d-ec8e-4238-9e8f-d35a722b3b39" (UID: "b58aec8d-ec8e-4238-9e8f-d35a722b3b39"). InnerVolumeSpecName "kube-api-access-d8wf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:11:27.254597 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.254540 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1107f52-46b9-40cb-9865-1f5aa38e6717-kube-api-access-6msss" (OuterVolumeSpecName: "kube-api-access-6msss") pod "b1107f52-46b9-40cb-9865-1f5aa38e6717" (UID: "b1107f52-46b9-40cb-9865-1f5aa38e6717"). InnerVolumeSpecName "kube-api-access-6msss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:11:27.294857 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.294828 2584 generic.go:356] "Generic (PLEG): container finished" podID="b58aec8d-ec8e-4238-9e8f-d35a722b3b39" containerID="4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a" exitCode=0 Sep 10 15:11:27.294997 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.294865 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" event={"ID":"b58aec8d-ec8e-4238-9e8f-d35a722b3b39","Type":"ContainerDied","Data":"4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a"} Sep 10 15:11:27.294997 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.294919 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" event={"ID":"b58aec8d-ec8e-4238-9e8f-d35a722b3b39","Type":"ContainerDied","Data":"6fad2facb5168cf6dd74f38e3d5ac85d34c6fdfa6f0518f675322a40d10b784e"} Sep 10 15:11:27.294997 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.294873 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h" Sep 10 15:11:27.294997 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.294941 2584 scope.go:117] "RemoveContainer" containerID="4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a" Sep 10 15:11:27.296217 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.296178 2584 generic.go:356] "Generic (PLEG): container finished" podID="b1107f52-46b9-40cb-9865-1f5aa38e6717" containerID="8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd" exitCode=0 Sep 10 15:11:27.296320 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.296240 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" Sep 10 15:11:27.296320 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.296237 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" event={"ID":"b1107f52-46b9-40cb-9865-1f5aa38e6717","Type":"ContainerDied","Data":"8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd"} Sep 10 15:11:27.296451 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.296350 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg" event={"ID":"b1107f52-46b9-40cb-9865-1f5aa38e6717","Type":"ContainerDied","Data":"5b964dd6ff30d8f044c27f7430f573dea3e99da23084b888067fb8b7c5553f3e"} Sep 10 15:11:27.305145 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.305125 2584 scope.go:117] "RemoveContainer" containerID="4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a" Sep 10 15:11:27.305421 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:11:27.305401 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a\": container with ID starting with 4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a not found: ID does not exist" containerID="4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a" Sep 10 15:11:27.305507 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.305429 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a"} err="failed to get container status \"4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a\": rpc error: code = NotFound desc = could not find container \"4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a\": container with ID starting with 4e6991557c104cc1151f0400b2bd83c5b3472b48fe8b1f9433b5076ca853967a not found: ID does not exist" Sep 10 15:11:27.305507 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.305444 2584 scope.go:117] "RemoveContainer" containerID="8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd" Sep 10 15:11:27.315780 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.314305 2584 scope.go:117] "RemoveContainer" containerID="8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd" Sep 10 15:11:27.315780 ip-10-0-136-202 kubenswrapper[2584]: E0910 15:11:27.314806 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd\": container with ID starting with 8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd not found: ID does not exist" containerID="8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd" Sep 10 15:11:27.315780 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.314835 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd"} err="failed to get container status \"8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd\": rpc error: code = NotFound desc = could not find container \"8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd\": container with ID starting with 8695f5c32696136cd72c1a567c065e26e66858b075bab8a710d408849babc3bd not found: ID does not exist" Sep 10 15:11:27.324285 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.324265 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h"] Sep 10 15:11:27.327912 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.327893 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3f81b7-5fd6c7d778-djs2h"] Sep 10 15:11:27.341730 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.341707 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg"] Sep 10 15:11:27.345228 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.345191 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-3f81b7-7b7cc79cbc-7x4gg"] Sep 10 15:11:27.353861 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.353844 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6msss\" (UniqueName: \"kubernetes.io/projected/b1107f52-46b9-40cb-9865-1f5aa38e6717-kube-api-access-6msss\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:27.353927 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.353863 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8wf4\" (UniqueName: \"kubernetes.io/projected/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-kube-api-access-d8wf4\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:27.353927 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.353877 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1107f52-46b9-40cb-9865-1f5aa38e6717-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:27.353927 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.353891 2584 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b58aec8d-ec8e-4238-9e8f-d35a722b3b39-plugins-config-volume\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:11:27.762118 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.762081 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24c7a886-c156-465b-8294-d293a538a349" path="/var/lib/kubelet/pods/24c7a886-c156-465b-8294-d293a538a349/volumes" Sep 10 15:11:27.762469 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.762455 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1107f52-46b9-40cb-9865-1f5aa38e6717" path="/var/lib/kubelet/pods/b1107f52-46b9-40cb-9865-1f5aa38e6717/volumes" Sep 10 15:11:27.762772 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:11:27.762760 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b58aec8d-ec8e-4238-9e8f-d35a722b3b39" path="/var/lib/kubelet/pods/b58aec8d-ec8e-4238-9e8f-d35a722b3b39/volumes" Sep 10 15:12:23.848985 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:12:23.848953 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:12:23.857255 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:12:23.857230 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:16:51.157039 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.156952 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2"] Sep 10 15:16:51.157608 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157503 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b58aec8d-ec8e-4238-9e8f-d35a722b3b39" containerName="payload-pre-processing" Sep 10 15:16:51.157608 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157548 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b58aec8d-ec8e-4238-9e8f-d35a722b3b39" containerName="payload-pre-processing" Sep 10 15:16:51.157608 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157567 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1107f52-46b9-40cb-9865-1f5aa38e6717" containerName="payload-processing" Sep 10 15:16:51.157608 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157577 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1107f52-46b9-40cb-9865-1f5aa38e6717" containerName="payload-processing" Sep 10 15:16:51.157608 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157590 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24c7a886-c156-465b-8294-d293a538a349" containerName="revoke-keys" Sep 10 15:16:51.157608 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157600 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="24c7a886-c156-465b-8294-d293a538a349" containerName="revoke-keys" Sep 10 15:16:51.157921 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157682 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1107f52-46b9-40cb-9865-1f5aa38e6717" containerName="payload-processing" Sep 10 15:16:51.157921 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157698 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="b58aec8d-ec8e-4238-9e8f-d35a722b3b39" containerName="payload-pre-processing" Sep 10 15:16:51.157921 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.157708 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="24c7a886-c156-465b-8294-d293a538a349" containerName="revoke-keys" Sep 10 15:16:51.160875 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.160852 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.163922 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.163900 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 10 15:16:51.165387 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.165365 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 10 15:16:51.165506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.165395 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 10 15:16:51.165506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.165395 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 10 15:16:51.165506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.165365 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9t8qm\"" Sep 10 15:16:51.173221 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.173172 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2"] Sep 10 15:16:51.314186 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.314140 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-metrics-certs\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.314414 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.314266 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6004e8dd-78cd-48a0-8168-44df9a1cea18-openshift-service-ca\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.314414 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.314297 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7lxq\" (UniqueName: \"kubernetes.io/projected/6004e8dd-78cd-48a0-8168-44df9a1cea18-kube-api-access-l7lxq\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.314550 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.314405 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-maas-api-tls\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.415696 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.415603 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-metrics-certs\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.415696 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.415657 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6004e8dd-78cd-48a0-8168-44df9a1cea18-openshift-service-ca\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.415696 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.415673 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7lxq\" (UniqueName: \"kubernetes.io/projected/6004e8dd-78cd-48a0-8168-44df9a1cea18-kube-api-access-l7lxq\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.415980 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.415709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-maas-api-tls\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.416303 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.416281 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6004e8dd-78cd-48a0-8168-44df9a1cea18-openshift-service-ca\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.418217 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.418177 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-metrics-certs\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.418300 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.418289 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-maas-api-tls\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.426863 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.426836 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7lxq\" (UniqueName: \"kubernetes.io/projected/6004e8dd-78cd-48a0-8168-44df9a1cea18-kube-api-access-l7lxq\") pod \"maas-api-7f975847c4-nkvw2\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.473826 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.473798 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:51.750313 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.750289 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2"] Sep 10 15:16:51.752300 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:16:51.752271 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6004e8dd_78cd_48a0_8168_44df9a1cea18.slice/crio-407da49d815781d9f07792b98fcd277bfbc02a81ba32158faeceda23f2bbebef WatchSource:0}: Error finding container 407da49d815781d9f07792b98fcd277bfbc02a81ba32158faeceda23f2bbebef: Status 404 returned error can't find the container with id 407da49d815781d9f07792b98fcd277bfbc02a81ba32158faeceda23f2bbebef Sep 10 15:16:51.753750 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.753733 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 10 15:16:51.838126 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:51.838094 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" event={"ID":"6004e8dd-78cd-48a0-8168-44df9a1cea18","Type":"ContainerStarted","Data":"407da49d815781d9f07792b98fcd277bfbc02a81ba32158faeceda23f2bbebef"} Sep 10 15:16:52.845012 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:52.844975 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" event={"ID":"6004e8dd-78cd-48a0-8168-44df9a1cea18","Type":"ContainerStarted","Data":"63b0f4b166fda2ab57a7560d566a544ed29a9c768fba8159d64c4894e8c22ef3"} Sep 10 15:16:52.845466 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:52.845103 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:52.866561 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:52.866512 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" podStartSLOduration=1.354536903 podStartE2EDuration="1.866498071s" totalImagesPullingTime="511.961168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:52.8643584 +0000 UTC m=+2069.536384183" watchObservedRunningTime="2026-09-10 15:16:52.866498071 +0000 UTC m=+2069.538523843" Sep 10 15:16:58.720653 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.720617 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2"] Sep 10 15:16:58.721085 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.720792 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" podUID="6004e8dd-78cd-48a0-8168-44df9a1cea18" containerName="maas-api" containerID="cri-o://63b0f4b166fda2ab57a7560d566a544ed29a9c768fba8159d64c4894e8c22ef3" gracePeriod=30 Sep 10 15:16:58.726067 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.726044 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:58.760014 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.759987 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj"] Sep 10 15:16:58.763381 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.763361 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.778331 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.778299 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj"] Sep 10 15:16:58.780387 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.780363 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a203fd6e-ce87-4df1-a352-b73ea18c3c37-openshift-service-ca\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.780519 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.780436 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czw5\" (UniqueName: \"kubernetes.io/projected/a203fd6e-ce87-4df1-a352-b73ea18c3c37-kube-api-access-5czw5\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.780519 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.780487 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a203fd6e-ce87-4df1-a352-b73ea18c3c37-metrics-certs\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.780639 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.780551 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a203fd6e-ce87-4df1-a352-b73ea18c3c37-maas-api-tls\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.874632 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.874596 2584 generic.go:356] "Generic (PLEG): container finished" podID="6004e8dd-78cd-48a0-8168-44df9a1cea18" containerID="63b0f4b166fda2ab57a7560d566a544ed29a9c768fba8159d64c4894e8c22ef3" exitCode=0 Sep 10 15:16:58.874797 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.874665 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" event={"ID":"6004e8dd-78cd-48a0-8168-44df9a1cea18","Type":"ContainerDied","Data":"63b0f4b166fda2ab57a7560d566a544ed29a9c768fba8159d64c4894e8c22ef3"} Sep 10 15:16:58.881012 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.880988 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a203fd6e-ce87-4df1-a352-b73ea18c3c37-openshift-service-ca\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.881173 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.881035 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czw5\" (UniqueName: \"kubernetes.io/projected/a203fd6e-ce87-4df1-a352-b73ea18c3c37-kube-api-access-5czw5\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.881173 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.881084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a203fd6e-ce87-4df1-a352-b73ea18c3c37-metrics-certs\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.881173 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.881119 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a203fd6e-ce87-4df1-a352-b73ea18c3c37-maas-api-tls\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.881683 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.881660 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a203fd6e-ce87-4df1-a352-b73ea18c3c37-openshift-service-ca\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.883610 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.883590 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a203fd6e-ce87-4df1-a352-b73ea18c3c37-maas-api-tls\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.883692 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.883672 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a203fd6e-ce87-4df1-a352-b73ea18c3c37-metrics-certs\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.890563 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.890542 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czw5\" (UniqueName: \"kubernetes.io/projected/a203fd6e-ce87-4df1-a352-b73ea18c3c37-kube-api-access-5czw5\") pod \"maas-api-6d6c5d7bdb-zrgmj\" (UID: \"a203fd6e-ce87-4df1-a352-b73ea18c3c37\") " pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:58.969899 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.969871 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:58.981993 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.981920 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-metrics-certs\") pod \"6004e8dd-78cd-48a0-8168-44df9a1cea18\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " Sep 10 15:16:58.982143 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.982000 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7lxq\" (UniqueName: \"kubernetes.io/projected/6004e8dd-78cd-48a0-8168-44df9a1cea18-kube-api-access-l7lxq\") pod \"6004e8dd-78cd-48a0-8168-44df9a1cea18\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " Sep 10 15:16:58.982143 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.982028 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-maas-api-tls\") pod \"6004e8dd-78cd-48a0-8168-44df9a1cea18\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " Sep 10 15:16:58.982143 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.982063 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6004e8dd-78cd-48a0-8168-44df9a1cea18-openshift-service-ca\") pod \"6004e8dd-78cd-48a0-8168-44df9a1cea18\" (UID: \"6004e8dd-78cd-48a0-8168-44df9a1cea18\") " Sep 10 15:16:58.982473 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.982452 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6004e8dd-78cd-48a0-8168-44df9a1cea18-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6004e8dd-78cd-48a0-8168-44df9a1cea18" (UID: "6004e8dd-78cd-48a0-8168-44df9a1cea18"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 10 15:16:58.984085 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.984065 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6004e8dd-78cd-48a0-8168-44df9a1cea18" (UID: "6004e8dd-78cd-48a0-8168-44df9a1cea18"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:16:58.984168 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.984105 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6004e8dd-78cd-48a0-8168-44df9a1cea18" (UID: "6004e8dd-78cd-48a0-8168-44df9a1cea18"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 10 15:16:58.984168 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:58.984116 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6004e8dd-78cd-48a0-8168-44df9a1cea18-kube-api-access-l7lxq" (OuterVolumeSpecName: "kube-api-access-l7lxq") pod "6004e8dd-78cd-48a0-8168-44df9a1cea18" (UID: "6004e8dd-78cd-48a0-8168-44df9a1cea18"). InnerVolumeSpecName "kube-api-access-l7lxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 10 15:16:59.082864 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.082822 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7lxq\" (UniqueName: \"kubernetes.io/projected/6004e8dd-78cd-48a0-8168-44df9a1cea18-kube-api-access-l7lxq\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:16:59.082864 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.082862 2584 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-maas-api-tls\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:16:59.083083 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.082875 2584 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6004e8dd-78cd-48a0-8168-44df9a1cea18-openshift-service-ca\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:16:59.083083 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.082887 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6004e8dd-78cd-48a0-8168-44df9a1cea18-metrics-certs\") on node \"ip-10-0-136-202.ec2.internal\" DevicePath \"\"" Sep 10 15:16:59.113608 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.113578 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:59.388576 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.388542 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj"] Sep 10 15:16:59.391346 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:16:59.391319 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda203fd6e_ce87_4df1_a352_b73ea18c3c37.slice/crio-b81a9d8c53b5b9064af0e53b2ab9acbfb9eee790585fa112a37b3959802a930d WatchSource:0}: Error finding container b81a9d8c53b5b9064af0e53b2ab9acbfb9eee790585fa112a37b3959802a930d: Status 404 returned error can't find the container with id b81a9d8c53b5b9064af0e53b2ab9acbfb9eee790585fa112a37b3959802a930d Sep 10 15:16:59.882977 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.882941 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" event={"ID":"a203fd6e-ce87-4df1-a352-b73ea18c3c37","Type":"ContainerStarted","Data":"bdaa56b47915bc12a835753e5d68766ace6453ae58d6bab95528845f857dd609"} Sep 10 15:16:59.883527 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.883002 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" event={"ID":"a203fd6e-ce87-4df1-a352-b73ea18c3c37","Type":"ContainerStarted","Data":"b81a9d8c53b5b9064af0e53b2ab9acbfb9eee790585fa112a37b3959802a930d"} Sep 10 15:16:59.883527 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.883050 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:16:59.884679 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.884654 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" event={"ID":"6004e8dd-78cd-48a0-8168-44df9a1cea18","Type":"ContainerDied","Data":"407da49d815781d9f07792b98fcd277bfbc02a81ba32158faeceda23f2bbebef"} Sep 10 15:16:59.884679 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.884673 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2" Sep 10 15:16:59.884879 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.884710 2584 scope.go:117] "RemoveContainer" containerID="63b0f4b166fda2ab57a7560d566a544ed29a9c768fba8159d64c4894e8c22ef3" Sep 10 15:16:59.904642 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.904583 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" podStartSLOduration=1.60787205 podStartE2EDuration="1.904562857s" totalImagesPullingTime="296.690807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:16:59.902688587 +0000 UTC m=+2076.574714359" watchObservedRunningTime="2026-09-10 15:16:59.904562857 +0000 UTC m=+2076.576588630" Sep 10 15:16:59.917674 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.917644 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2"] Sep 10 15:16:59.920818 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:16:59.920794 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f975847c4-nkvw2"] Sep 10 15:17:01.762980 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:01.762899 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6004e8dd-78cd-48a0-8168-44df9a1cea18" path="/var/lib/kubelet/pods/6004e8dd-78cd-48a0-8168-44df9a1cea18/volumes" Sep 10 15:17:05.896883 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:05.896847 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d6c5d7bdb-zrgmj" Sep 10 15:17:23.884303 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:23.884276 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:17:23.891301 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:23.891280 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5z54x_4365ce51-f8bc-4b0f-9864-e9309738013f/ovn-acl-logging/0.log" Sep 10 15:17:24.049494 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:24.049458 2584 scope.go:117] "RemoveContainer" containerID="e484d7ad51b3427533ad1ddfbcc2817574d535ad12afcc570401a13150a04e5b" Sep 10 15:17:24.057326 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:24.057245 2584 scope.go:117] "RemoveContainer" containerID="87fc6b4ca8a754683a0d54b80aeb8108e0500b791bdf1d9387f7d9069c32514f" Sep 10 15:17:24.064536 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:24.064511 2584 scope.go:117] "RemoveContainer" containerID="f867a06e0447b7b1cc57f18a5f1b6a4a803dd1cb266dd78e13ad3ea6876aab78" Sep 10 15:17:26.869720 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:26.869681 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wk6mv_06adc0df-7271-4324-ba26-94e3847000e9/server/0.log" Sep 10 15:17:26.997949 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:26.997912 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dhxjp_c2da0e7e-ad04-4157-ac82-8bbb34026a15/manager/0.log" Sep 10 15:17:27.118617 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:27.118591 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-258zd_d0e87f50-d14f-4e0c-8b7d-94b16320033a/copy-manifests/0.log" Sep 10 15:17:27.126632 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:27.125439 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-258zd_d0e87f50-d14f-4e0c-8b7d-94b16320033a/manager/0.log" Sep 10 15:17:27.819467 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:27.819430 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6d6c5d7bdb-zrgmj_a203fd6e-ce87-4df1-a352-b73ea18c3c37/maas-api/0.log" Sep 10 15:17:27.928885 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:27.928846 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817540-rmdhh_45eda6db-975b-41a5-baae-fd09bb4fa5e5/cleanup/0.log" Sep 10 15:17:29.993037 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:29.993008 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc_552a922a-babc-46d8-8e08-e59ee9704a80/istio-proxy/0.log" Sep 10 15:17:30.110472 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:30.110440 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fhwjc_5538635f-c690-4e2a-aaf8-1fdfc90b7059/discovery/0.log" Sep 10 15:17:30.447287 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:30.447234 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rbmhd_a1509437-33ca-470a-8b3b-4078375ad640/istio-proxy/1.log" Sep 10 15:17:30.566852 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:30.566810 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2rh5c_3425b6dc-5b64-42b6-991d-d645eef00447/payload-pre-processing/0.log" Sep 10 15:17:30.681545 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:30.681513 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scfq4_e92eeed2-5be2-46ad-8c24-65ef2f89b0c8/payload-processing/0.log" Sep 10 15:17:31.574836 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:31.574804 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fgs6l7_9159abab-61a8-4a2c-8182-dd11ecce2f86/main/0.log" Sep 10 15:17:31.686027 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:31.685998 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rrlrl_c999073a-2d10-4a8b-a4df-35dd4bebde68/main/0.log" Sep 10 15:17:40.153552 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:40.153523 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zlhxp_01b92fcc-fb48-4ed4-9bf8-86594f042671/global-pull-secret-syncer/0.log" Sep 10 15:17:40.232881 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:40.232853 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p2h4g_65b1d81e-eb65-41f4-b525-4cc3ec113c55/konnectivity-agent/0.log" Sep 10 15:17:40.292806 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:40.292778 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-202.ec2.internal_0f8f991da7ffc336926674c917982fec/haproxy/0.log" Sep 10 15:17:47.525735 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.525663 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l652x_623acfa3-a4c4-415a-ad9f-13c298fd6b42/node-exporter/0.log" Sep 10 15:17:47.558997 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.558972 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l652x_623acfa3-a4c4-415a-ad9f-13c298fd6b42/kube-rbac-proxy/0.log" Sep 10 15:17:47.581118 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.581093 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l652x_623acfa3-a4c4-415a-ad9f-13c298fd6b42/init-textfile/0.log" Sep 10 15:17:47.689610 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.689576 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3d147cbe-24f9-4db8-86aa-c3ebd7b9817b/prometheus/0.log" Sep 10 15:17:47.709506 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.709474 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3d147cbe-24f9-4db8-86aa-c3ebd7b9817b/config-reloader/0.log" Sep 10 15:17:47.734990 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.734964 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3d147cbe-24f9-4db8-86aa-c3ebd7b9817b/thanos-sidecar/0.log" Sep 10 15:17:47.756840 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.756817 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3d147cbe-24f9-4db8-86aa-c3ebd7b9817b/kube-rbac-proxy-web/0.log" Sep 10 15:17:47.778523 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.778454 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3d147cbe-24f9-4db8-86aa-c3ebd7b9817b/kube-rbac-proxy/0.log" Sep 10 15:17:47.799756 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.799732 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3d147cbe-24f9-4db8-86aa-c3ebd7b9817b/kube-rbac-proxy-thanos/0.log" Sep 10 15:17:47.822662 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.822634 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3d147cbe-24f9-4db8-86aa-c3ebd7b9817b/init-config-reloader/0.log" Sep 10 15:17:47.927535 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:47.927509 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-7r2qm_3b23bc10-cb9e-4a5e-8454-4b2ce0c5cba2/prometheus-operator-admission-webhook/0.log" Sep 10 15:17:49.542574 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:49.542544 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-ljjnd_ffce56c3-370a-44b9-a9b0-5d51b1fe63d2/networking-console-plugin/0.log" Sep 10 15:17:50.319547 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.319515 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz"] Sep 10 15:17:50.319859 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.319848 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6004e8dd-78cd-48a0-8168-44df9a1cea18" containerName="maas-api" Sep 10 15:17:50.319906 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.319861 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6004e8dd-78cd-48a0-8168-44df9a1cea18" containerName="maas-api" Sep 10 15:17:50.319941 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.319928 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6004e8dd-78cd-48a0-8168-44df9a1cea18" containerName="maas-api" Sep 10 15:17:50.323196 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.323171 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.327067 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.327035 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-prcmw\"/\"default-dockercfg-9v44h\"" Sep 10 15:17:50.327067 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.327067 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prcmw\"/\"kube-root-ca.crt\"" Sep 10 15:17:50.328139 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.328121 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prcmw\"/\"openshift-service-ca.crt\"" Sep 10 15:17:50.334846 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.334822 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz"] Sep 10 15:17:50.355517 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.355485 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-sys\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.355646 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.355585 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slrgw\" (UniqueName: \"kubernetes.io/projected/e9b4abf2-5187-406a-a556-a4cc05254566-kube-api-access-slrgw\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.355646 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.355606 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-proc\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.355646 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.355641 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-podres\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.355777 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.355660 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-lib-modules\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.456809 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456774 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slrgw\" (UniqueName: \"kubernetes.io/projected/e9b4abf2-5187-406a-a556-a4cc05254566-kube-api-access-slrgw\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.456809 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456810 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-proc\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.457000 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456836 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-podres\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.457000 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456864 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-lib-modules\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.457000 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456912 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-sys\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.457000 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456917 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-proc\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.457000 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456968 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-podres\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.457000 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.456980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-sys\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.457236 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.457029 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9b4abf2-5187-406a-a556-a4cc05254566-lib-modules\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.466180 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.466148 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slrgw\" (UniqueName: \"kubernetes.io/projected/e9b4abf2-5187-406a-a556-a4cc05254566-kube-api-access-slrgw\") pod \"perf-node-gather-daemonset-j8hqz\" (UID: \"e9b4abf2-5187-406a-a556-a4cc05254566\") " pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.634887 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.634802 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:50.740949 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.740918 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-lzdq5_f0d8a303-d2bd-42e8-8b80-713a09fe9195/download-server/0.log" Sep 10 15:17:50.908596 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:50.908571 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz"] Sep 10 15:17:50.910817 ip-10-0-136-202 kubenswrapper[2584]: W0910 15:17:50.910788 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode9b4abf2_5187_406a_a556_a4cc05254566.slice/crio-9c6f401a427e38f99c5804d16194dad39261f8d84974fcb357e1c06c3fd706b4 WatchSource:0}: Error finding container 9c6f401a427e38f99c5804d16194dad39261f8d84974fcb357e1c06c3fd706b4: Status 404 returned error can't find the container with id 9c6f401a427e38f99c5804d16194dad39261f8d84974fcb357e1c06c3fd706b4 Sep 10 15:17:51.132532 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:51.132435 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" event={"ID":"e9b4abf2-5187-406a-a556-a4cc05254566","Type":"ContainerStarted","Data":"b883c50a696c55551da8172fa023cf55720b51ead43a9db5101b1cd2812c5ab6"} Sep 10 15:17:51.132532 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:51.132501 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" event={"ID":"e9b4abf2-5187-406a-a556-a4cc05254566","Type":"ContainerStarted","Data":"9c6f401a427e38f99c5804d16194dad39261f8d84974fcb357e1c06c3fd706b4"} Sep 10 15:17:51.132822 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:51.132597 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:51.151915 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:51.151867 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" podStartSLOduration=1.151852991 podStartE2EDuration="1.151852991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:17:51.149843153 +0000 UTC m=+2127.821868923" watchObservedRunningTime="2026-09-10 15:17:51.151852991 +0000 UTC m=+2127.823878799" Sep 10 15:17:52.612821 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:52.612791 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m5fkb_e5e5998f-51ba-4af7-8f00-334c457bd7fa/dns/0.log" Sep 10 15:17:52.633411 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:52.633384 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m5fkb_e5e5998f-51ba-4af7-8f00-334c457bd7fa/kube-rbac-proxy/0.log" Sep 10 15:17:52.730559 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:52.730534 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qb7fw_416d5edd-4a50-4621-947f-46bffc5cb8c0/dns-node-resolver/0.log" Sep 10 15:17:53.302837 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:53.302806 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-858d94df59-2fccl_943cf98b-80ce-43ba-bb4f-40ad7fcde845/registry/0.log" Sep 10 15:17:53.326573 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:53.326544 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6mpq9_cfcf1f64-077b-471c-8cba-0f0a90180acd/node-ca/0.log" Sep 10 15:17:54.357914 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:54.357887 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc_552a922a-babc-46d8-8e08-e59ee9704a80/istio-proxy/0.log" Sep 10 15:17:54.545012 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:54.544984 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fhwjc_5538635f-c690-4e2a-aaf8-1fdfc90b7059/discovery/0.log" Sep 10 15:17:54.652235 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:54.652121 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rbmhd_a1509437-33ca-470a-8b3b-4078375ad640/istio-proxy/0.log" Sep 10 15:17:54.659465 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:54.659434 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rbmhd_a1509437-33ca-470a-8b3b-4078375ad640/istio-proxy/1.log" Sep 10 15:17:54.694746 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:54.694718 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2rh5c_3425b6dc-5b64-42b6-991d-d645eef00447/payload-pre-processing/0.log" Sep 10 15:17:54.731245 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:54.731222 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scfq4_e92eeed2-5be2-46ad-8c24-65ef2f89b0c8/payload-processing/0.log" Sep 10 15:17:55.594077 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:55.594050 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6jvfq_b303f07a-2785-4c69-bd70-6ed299b78443/serve-healthcheck-canary/0.log" Sep 10 15:17:56.376518 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:56.376484 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zmw6z_f78edbc3-32a9-45ba-8160-cab3879b0ff8/kube-rbac-proxy/0.log" Sep 10 15:17:56.403225 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:56.400011 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zmw6z_f78edbc3-32a9-45ba-8160-cab3879b0ff8/exporter/0.log" Sep 10 15:17:56.423334 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:56.423307 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zmw6z_f78edbc3-32a9-45ba-8160-cab3879b0ff8/extractor/0.log" Sep 10 15:17:57.147183 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:57.147145 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-prcmw/perf-node-gather-daemonset-j8hqz" Sep 10 15:17:58.789105 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:58.789076 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-wk6mv_06adc0df-7271-4324-ba26-94e3847000e9/server/0.log" Sep 10 15:17:58.866321 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:58.866295 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dhxjp_c2da0e7e-ad04-4157-ac82-8bbb34026a15/manager/0.log" Sep 10 15:17:58.894389 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:58.894363 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-258zd_d0e87f50-d14f-4e0c-8b7d-94b16320033a/manager/0.log" Sep 10 15:17:58.916218 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:17:58.916179 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-68f84f89d7-258zd_d0e87f50-d14f-4e0c-8b7d-94b16320033a/copy-manifests/0.log" Sep 10 15:18:00.548355 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:00.548325 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bc768b6-8wjxv_1c1351a7-f82b-4bbd-bf6a-31000e0c8843/manager/0.log" Sep 10 15:18:00.604821 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:00.604764 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-7r98n_354f19fe-8c46-4c56-9b18-f6ac68b07588/openshift-lws-operator/0.log" Sep 10 15:18:00.938251 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:00.938221 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc_552a922a-babc-46d8-8e08-e59ee9704a80/istio-proxy/0.log" Sep 10 15:18:01.140629 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:01.140595 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fhwjc_5538635f-c690-4e2a-aaf8-1fdfc90b7059/discovery/0.log" Sep 10 15:18:01.189980 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:01.189888 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rbmhd_a1509437-33ca-470a-8b3b-4078375ad640/istio-proxy/0.log" Sep 10 15:18:01.197144 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:01.197118 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rbmhd_a1509437-33ca-470a-8b3b-4078375ad640/istio-proxy/1.log" Sep 10 15:18:01.224477 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:01.224445 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2rh5c_3425b6dc-5b64-42b6-991d-d645eef00447/payload-pre-processing/0.log" Sep 10 15:18:01.252561 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:01.252529 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scfq4_e92eeed2-5be2-46ad-8c24-65ef2f89b0c8/payload-processing/0.log" Sep 10 15:18:02.714717 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:02.714675 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c9j8cc_552a922a-babc-46d8-8e08-e59ee9704a80/istio-proxy/0.log" Sep 10 15:18:02.922613 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:02.922568 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-fhwjc_5538635f-c690-4e2a-aaf8-1fdfc90b7059/discovery/0.log" Sep 10 15:18:02.979047 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:02.978924 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rbmhd_a1509437-33ca-470a-8b3b-4078375ad640/istio-proxy/0.log" Sep 10 15:18:02.986031 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:02.986005 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rbmhd_a1509437-33ca-470a-8b3b-4078375ad640/istio-proxy/1.log" Sep 10 15:18:03.011590 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:03.011530 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2rh5c_3425b6dc-5b64-42b6-991d-d645eef00447/payload-pre-processing/0.log" Sep 10 15:18:03.037128 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:03.037083 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scfq4_e92eeed2-5be2-46ad-8c24-65ef2f89b0c8/payload-processing/0.log" Sep 10 15:18:05.738661 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:05.738627 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-wrtfd_eacd8d86-9eb3-48b8-a818-0b011faad98b/migrator/0.log" Sep 10 15:18:05.768284 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:05.768252 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-wrtfd_eacd8d86-9eb3-48b8-a818-0b011faad98b/graceful-termination/0.log" Sep 10 15:18:07.518021 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.517992 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jfwfh_11e4848c-8666-4f5a-bc95-d0e5c8cc5154/kube-multus-additional-cni-plugins/0.log" Sep 10 15:18:07.540920 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.540866 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jfwfh_11e4848c-8666-4f5a-bc95-d0e5c8cc5154/egress-router-binary-copy/0.log" Sep 10 15:18:07.562451 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.562422 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jfwfh_11e4848c-8666-4f5a-bc95-d0e5c8cc5154/cni-plugins/0.log" Sep 10 15:18:07.586274 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.586195 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jfwfh_11e4848c-8666-4f5a-bc95-d0e5c8cc5154/bond-cni-plugin/0.log" Sep 10 15:18:07.620471 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.620438 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jfwfh_11e4848c-8666-4f5a-bc95-d0e5c8cc5154/routeoverride-cni/0.log" Sep 10 15:18:07.671096 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.671063 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jfwfh_11e4848c-8666-4f5a-bc95-d0e5c8cc5154/whereabouts-cni-bincopy/0.log" Sep 10 15:18:07.695106 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.695076 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jfwfh_11e4848c-8666-4f5a-bc95-d0e5c8cc5154/whereabouts-cni/0.log" Sep 10 15:18:07.743770 ip-10-0-136-202 kubenswrapper[2584]: I0910 15:18:07.743738 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pc9td_05fbf59b-3045-4815-aae2-2cae55138ab0/kube-multus/0.log"