Sep 07 08:57:01.364085 ip-10-0-131-66 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 07 08:57:01.364097 ip-10-0-131-66 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 07 08:57:01.364107 ip-10-0-131-66 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 07 08:57:01.364315 ip-10-0-131-66 systemd[1]: Failed to start Kubernetes Kubelet. Sep 07 08:57:11.495024 ip-10-0-131-66 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 07 08:57:11.495044 ip-10-0-131-66 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 11f9e8dd16f7432f8adf45ed99e4cc3e -- Sep 07 08:59:00.099661 ip-10-0-131-66 systemd[1]: Starting Kubernetes Kubelet... Sep 07 08:59:00.458127 ip-10-0-131-66 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 08:59:00.458127 ip-10-0-131-66 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 07 08:59:00.458127 ip-10-0-131-66 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 08:59:00.458127 ip-10-0-131-66 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 07 08:59:00.458127 ip-10-0-131-66 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 07 08:59:00.459183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.459095 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 07 08:59:00.463457 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463443 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 08:59:00.463457 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463457 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463461 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463470 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463473 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463477 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463480 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463483 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463486 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463489 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463491 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463494 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463497 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463499 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463502 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463504 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463507 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463510 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463512 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463515 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463517 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 08:59:00.463523 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463519 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463522 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463525 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463527 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463530 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463533 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463536 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463539 2558 feature_gate.go:328] unrecognized feature gate: Example Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463541 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463544 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463552 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463556 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463558 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463560 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463563 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463567 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463570 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463573 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463576 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 08:59:00.464024 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463579 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463582 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463584 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463587 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463590 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463595 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463598 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463601 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463603 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463606 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463609 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463612 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463614 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463617 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463620 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463623 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463626 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463628 2558 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463631 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463633 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 08:59:00.464588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463636 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463638 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463641 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463643 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463651 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463654 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463657 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463659 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463662 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463664 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463667 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463669 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463672 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463674 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463677 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463679 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463683 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463686 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463690 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463693 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 08:59:00.465084 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463695 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463698 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463701 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463703 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463706 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463709 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463711 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463715 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463717 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463720 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463722 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463725 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463727 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463730 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463732 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463735 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463737 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463745 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.463748 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 08:59:00.465592 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465232 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465240 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465244 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465247 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465250 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465254 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465257 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465261 2558 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465263 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465266 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465269 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465271 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465274 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465277 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465280 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465284 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465286 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465289 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465292 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465294 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 08:59:00.466065 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465298 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465300 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465304 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465308 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465312 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465314 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465317 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465319 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465322 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465325 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465327 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465330 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465333 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465336 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465338 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465341 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465343 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465346 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465348 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465351 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 08:59:00.466613 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465354 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465356 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465371 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465373 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465376 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465379 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465382 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465384 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465387 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465390 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465392 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465395 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465398 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465401 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465404 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465406 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465409 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465413 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465416 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465418 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 08:59:00.467106 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465421 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465423 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465426 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465428 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465431 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465434 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465437 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465440 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465442 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465445 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465447 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465450 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465453 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465455 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465457 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465460 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465462 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465465 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465467 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465470 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 08:59:00.467621 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465473 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465475 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465478 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465480 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465487 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465489 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465492 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465494 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465497 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465499 2558 feature_gate.go:328] unrecognized feature gate: Example Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465502 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465505 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465507 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465510 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465512 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465515 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465517 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465520 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.465523 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466511 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 08:59:00.468116 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466518 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466522 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466526 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466528 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466531 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466533 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466536 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466539 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466541 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466544 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466546 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466549 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466551 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466554 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466557 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466559 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466563 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466566 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466568 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 08:59:00.468608 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466572 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466576 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466579 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466582 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466584 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466587 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466591 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466593 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466596 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466599 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466601 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466604 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466607 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466623 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466627 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466631 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466635 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466638 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466641 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466644 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 08:59:00.469083 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466647 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466650 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466653 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466655 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466658 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466661 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466664 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466667 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466670 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466673 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466676 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466678 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466681 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466683 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466686 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466688 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466691 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466693 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466696 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466698 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 08:59:00.469600 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466701 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466703 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466706 2558 feature_gate.go:328] unrecognized feature gate: Example Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466709 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466712 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466715 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466718 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466721 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466724 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466727 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466729 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466732 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466734 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466737 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466740 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466742 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466745 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466747 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466750 2558 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466752 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 08:59:00.470093 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466755 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466757 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466760 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466762 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466765 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466769 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466772 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466775 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466777 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466780 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466782 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466785 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466787 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466790 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466792 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466795 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466798 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466801 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.466804 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466864 2558 flags.go:64] FLAG: --address="0.0.0.0" Sep 07 08:59:00.470588 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466897 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466905 2558 flags.go:64] FLAG: --anonymous-auth="true" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466911 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466916 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466920 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466924 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466929 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466932 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466935 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466938 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466942 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466945 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466948 2558 flags.go:64] FLAG: --cgroup-root="" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466950 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466953 2558 flags.go:64] FLAG: --client-ca-file="" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466956 2558 flags.go:64] FLAG: --cloud-provider="external" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466959 2558 flags.go:64] FLAG: --cluster-dns="[]" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466964 2558 flags.go:64] FLAG: --cluster-domain="" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466967 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466970 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466973 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466976 2558 flags.go:64] FLAG: --container-log-max-files="5" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466980 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 07 08:59:00.471101 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466983 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466985 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466989 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466993 2558 flags.go:64] FLAG: --contention-profiling="false" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.466996 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467000 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467004 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467007 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467012 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467015 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467018 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467021 2558 flags.go:64] FLAG: --enable-load-reader="false" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467024 2558 flags.go:64] FLAG: --enable-server="true" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467027 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467031 2558 flags.go:64] FLAG: --event-burst="100" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467034 2558 flags.go:64] FLAG: --event-qps="50" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467037 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467041 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467044 2558 flags.go:64] FLAG: --eviction-hard="" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467047 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467050 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467053 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467056 2558 flags.go:64] FLAG: --eviction-soft="" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467059 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467062 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 07 08:59:00.472010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467064 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467067 2558 flags.go:64] FLAG: --experimental-mounter-path="" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467070 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467073 2558 flags.go:64] FLAG: --fail-swap-on="true" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467076 2558 flags.go:64] FLAG: --feature-gates="" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467079 2558 flags.go:64] FLAG: --file-check-frequency="20s" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467082 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467085 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467088 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467091 2558 flags.go:64] FLAG: --healthz-port="10248" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467095 2558 flags.go:64] FLAG: --help="false" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467098 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467101 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467104 2558 flags.go:64] FLAG: --http-check-frequency="20s" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467108 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467113 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467117 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467120 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467122 2558 flags.go:64] FLAG: --image-service-endpoint="" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467125 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467128 2558 flags.go:64] FLAG: --kube-api-burst="100" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467131 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467134 2558 flags.go:64] FLAG: --kube-api-qps="50" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467137 2558 flags.go:64] FLAG: --kube-reserved="" Sep 07 08:59:00.472749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467140 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467143 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467145 2558 flags.go:64] FLAG: --kubelet-cgroups="" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467148 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467151 2558 flags.go:64] FLAG: --lock-file="" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467154 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467157 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467160 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467165 2558 flags.go:64] FLAG: --log-json-split-stream="false" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467168 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467170 2558 flags.go:64] FLAG: --log-text-split-stream="false" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467173 2558 flags.go:64] FLAG: --logging-format="text" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467176 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467179 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467182 2558 flags.go:64] FLAG: --manifest-url="" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467185 2558 flags.go:64] FLAG: --manifest-url-header="" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467189 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467192 2558 flags.go:64] FLAG: --max-open-files="1000000" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467197 2558 flags.go:64] FLAG: --max-pods="110" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467201 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467203 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467206 2558 flags.go:64] FLAG: --memory-manager-policy="None" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467209 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467213 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467217 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 07 08:59:00.473342 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467220 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467226 2558 flags.go:64] FLAG: --node-status-max-images="50" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467229 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467232 2558 flags.go:64] FLAG: --oom-score-adj="-999" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467235 2558 flags.go:64] FLAG: --pod-cidr="" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467238 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467244 2558 flags.go:64] FLAG: --pod-manifest-path="" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467246 2558 flags.go:64] FLAG: --pod-max-pids="-1" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467249 2558 flags.go:64] FLAG: --pods-per-core="0" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467252 2558 flags.go:64] FLAG: --port="10250" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467255 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467258 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0857c419a86a60218" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467261 2558 flags.go:64] FLAG: --qos-reserved="" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467264 2558 flags.go:64] FLAG: --read-only-port="10255" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467267 2558 flags.go:64] FLAG: --register-node="true" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467270 2558 flags.go:64] FLAG: --register-with-taints="" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467273 2558 flags.go:64] FLAG: --registry-burst="10" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467276 2558 flags.go:64] FLAG: --registry-qps="5" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467279 2558 flags.go:64] FLAG: --reserved-cpus="" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467282 2558 flags.go:64] FLAG: --reserved-memory="" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467286 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467289 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467292 2558 flags.go:64] FLAG: --rotate-certificates="false" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467294 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467297 2558 flags.go:64] FLAG: --runonce="false" Sep 07 08:59:00.474081 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467300 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467304 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467307 2558 flags.go:64] FLAG: --seccomp-default="false" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467310 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467313 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467317 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467320 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467323 2558 flags.go:64] FLAG: --storage-driver-password="root" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467326 2558 flags.go:64] FLAG: --storage-driver-secure="false" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467329 2558 flags.go:64] FLAG: --storage-driver-table="stats" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467332 2558 flags.go:64] FLAG: --storage-driver-user="root" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467335 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467338 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467340 2558 flags.go:64] FLAG: --system-cgroups="" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467343 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467349 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467352 2558 flags.go:64] FLAG: --tls-cert-file="" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467354 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467376 2558 flags.go:64] FLAG: --tls-min-version="" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467379 2558 flags.go:64] FLAG: --tls-private-key-file="" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467382 2558 flags.go:64] FLAG: --topology-manager-policy="none" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467385 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467388 2558 flags.go:64] FLAG: --topology-manager-scope="container" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467390 2558 flags.go:64] FLAG: --v="2" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467394 2558 flags.go:64] FLAG: --version="false" Sep 07 08:59:00.474717 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467398 2558 flags.go:64] FLAG: --vmodule="" Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467402 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.467406 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467517 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467520 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467524 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467527 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467530 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467536 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467539 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467543 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467546 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467551 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467554 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467558 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467561 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467564 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467567 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467569 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 08:59:00.475335 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467572 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467574 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467577 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467580 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467582 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467585 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467587 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467590 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467592 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467595 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467598 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467600 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467602 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467605 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467607 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467610 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467612 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467615 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467617 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467620 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 08:59:00.475822 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467622 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467626 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467630 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467634 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467637 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467641 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467644 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467647 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467650 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467653 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467655 2558 feature_gate.go:328] unrecognized feature gate: Example Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467658 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467661 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467664 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467666 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467669 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467671 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467674 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467676 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 08:59:00.476344 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467679 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467681 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467684 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467686 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467689 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467692 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467694 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467697 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467699 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467702 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467704 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467707 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467709 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467712 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467715 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467718 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467721 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467723 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467729 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467732 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 08:59:00.476895 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467734 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467737 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467740 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467742 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467745 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467747 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467750 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467753 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467755 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467757 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467760 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467762 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467765 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467767 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467770 2558 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467772 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467775 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467777 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467780 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467782 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 08:59:00.477486 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467785 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 08:59:00.477979 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467787 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 08:59:00.477979 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467790 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 08:59:00.477979 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.467793 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 08:59:00.477979 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.469176 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 08:59:00.478249 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.478230 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 07 08:59:00.478281 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.478249 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 07 08:59:00.479156 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.479146 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 07 08:59:00.479198 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.479161 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 07 08:59:00.479229 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479217 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 08:59:00.479229 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479223 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 08:59:00.479229 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479226 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 08:59:00.479229 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479230 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479233 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479236 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479239 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479242 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479244 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479247 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479250 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479252 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479255 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479257 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479260 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479263 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479265 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479268 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479270 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479273 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479276 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479278 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479281 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 08:59:00.479336 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479284 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479286 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479289 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479292 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479299 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479303 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479306 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479308 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479311 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479314 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479316 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479319 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479321 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479324 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479328 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479330 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479333 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479336 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479339 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479342 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 08:59:00.479830 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479344 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479347 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479350 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479352 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479355 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479357 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479374 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479378 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479381 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479384 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479386 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479389 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479392 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479395 2558 feature_gate.go:328] unrecognized feature gate: Example Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479397 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479400 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479402 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479405 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479407 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479410 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 08:59:00.480310 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479413 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479415 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479418 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479421 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479423 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479426 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479429 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479433 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479437 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479439 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479442 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479445 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479447 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479450 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479453 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479455 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479459 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479461 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479464 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479467 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 08:59:00.480842 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479469 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479472 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479475 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479477 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479480 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479482 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479485 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479488 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479492 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479495 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479498 2558 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479501 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479504 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479507 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479510 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 08:59:00.481324 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479512 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.479517 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479628 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479633 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479637 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479640 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479643 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479645 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479648 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479650 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479655 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479660 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479663 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479666 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 07 08:59:00.481709 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479669 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479672 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479675 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479678 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479682 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479685 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479688 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479690 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479693 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479696 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479698 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479701 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479704 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479706 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479709 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479711 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479714 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479716 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479719 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479721 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 07 08:59:00.482061 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479724 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479726 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479730 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479732 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479734 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479737 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479740 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479742 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479745 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479748 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479750 2558 feature_gate.go:328] unrecognized feature gate: Example Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479753 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479756 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479758 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479761 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479763 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479766 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479768 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479771 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479773 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479776 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 07 08:59:00.482564 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479778 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479781 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479783 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479786 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479788 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479791 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479794 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479796 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479799 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479801 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479804 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479806 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479809 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479812 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479814 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479816 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479819 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479821 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479824 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479826 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 07 08:59:00.483069 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479829 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479831 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479834 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479837 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479839 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479842 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479845 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479848 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479850 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479852 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479855 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479857 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479860 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479862 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479865 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479868 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479870 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479873 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479875 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479878 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 07 08:59:00.483565 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479880 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479883 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479885 2558 feature_gate.go:328] unrecognized feature gate: Example2 Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479888 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479892 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:00.479895 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.479900 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.480010 2558 server.go:962] "Client rotation is on, will bootstrap in background" Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.482484 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.483562 2558 server.go:1019] "Starting client certificate rotation" Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.483655 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 08:59:00.484075 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.483688 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 07 08:59:00.486450 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.486346 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 08:59:00.487400 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.487383 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 07 08:59:00.502416 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.502396 2558 log.go:25] "Validated CRI v1 runtime API" Sep 07 08:59:00.507603 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.507585 2558 log.go:25] "Validated CRI v1 image API" Sep 07 08:59:00.509232 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.509211 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 07 08:59:00.513158 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.513139 2558 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 841ff72e-da1d-4bfe-b703-3c6d4889e906:/dev/nvme0n1p3 90283386-ade0-4fc1-9150-b0a51ea084a8:/dev/nvme0n1p4] Sep 07 08:59:00.513213 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.513157 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 07 08:59:00.516168 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.516150 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 08:59:00.519373 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.519257 2558 manager.go:216] Machine: {Timestamp:2026-09-07 08:59:00.516873787 +0000 UTC m=+0.315600006 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099953 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27f97c404965514679279e5648a956 SystemUUID:ec27f97c-4049-6551-4679-279e5648a956 BootID:11f9e8dd-16f7-432f-8adf-45ed99e4cc3e Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:66:07:03:df:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:66:07:03:df:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:4b:71:35:59:2f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 07 08:59:00.519373 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.519372 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 07 08:59:00.519487 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.519451 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 07 08:59:00.523199 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.523173 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 07 08:59:00.523328 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.523201 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-66.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 07 08:59:00.524080 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.524069 2558 topology_manager.go:138] "Creating topology manager with none policy" Sep 07 08:59:00.524118 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.524083 2558 container_manager_linux.go:309] "Creating device plugin manager" Sep 07 08:59:00.524118 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.524090 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 08:59:00.524933 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.524923 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 07 08:59:00.526935 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.526925 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 07 08:59:00.532482 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532470 2558 state_mem.go:36] "Initialized new in-memory state store" Sep 07 08:59:00.532592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532582 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 07 08:59:00.532643 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532613 2558 kubelet.go:486] "Attempting to sync node with API server" Sep 07 08:59:00.532643 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532623 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 07 08:59:00.532643 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532641 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 07 08:59:00.532742 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532651 2558 kubelet.go:398] "Adding apiserver pod source" Sep 07 08:59:00.532742 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532660 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 07 08:59:00.532996 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.532984 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 08:59:00.533047 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.533003 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 07 08:59:00.535505 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.535488 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 07 08:59:00.538349 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538336 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 07 08:59:00.538408 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538384 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 07 08:59:00.538557 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538547 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 07 08:59:00.538587 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538562 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 07 08:59:00.538587 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538568 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 07 08:59:00.538587 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538573 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 07 08:59:00.538587 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538581 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 07 08:59:00.538587 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.538587 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 07 08:59:00.539221 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.539208 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 07 08:59:00.539221 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.539219 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 07 08:59:00.539310 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.539226 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 07 08:59:00.539310 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.539231 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 07 08:59:00.539310 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.539241 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 07 08:59:00.539310 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.539249 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 07 08:59:00.540157 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.540140 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c9skv" Sep 07 08:59:00.541038 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.541029 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 07 08:59:00.541038 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.541038 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 07 08:59:00.544752 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.544738 2558 server.go:1268] "Started kubelet" Sep 07 08:59:00.544894 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.544852 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 07 08:59:00.544950 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.544922 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 07 08:59:00.545374 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.544818 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 07 08:59:00.545711 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.545695 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c9skv" Sep 07 08:59:00.545766 ip-10-0-131-66 systemd[1]: Started Kubernetes Kubelet. Sep 07 08:59:00.547540 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.547526 2558 server.go:310] "Adding debug handlers to kubelet server" Sep 07 08:59:00.547708 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.547692 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 07 08:59:00.549898 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.549863 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-66.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 07 08:59:00.550147 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.550130 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 07 08:59:00.550698 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.550671 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 07 08:59:00.550757 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.550692 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 07 08:59:00.554140 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.549837 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-66.ec2.internal.18d2fe3d227eea2b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-66.ec2.internal,UID:ip-10-0-131-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-66.ec2.internal,},FirstTimestamp:2026-09-07 08:59:00.544711211 +0000 UTC m=+0.343437429,LastTimestamp:2026-09-07 08:59:00.544711211 +0000 UTC m=+0.343437429,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-66.ec2.internal,}" Sep 07 08:59:00.554283 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.554253 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 07 08:59:00.555627 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.555613 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 07 08:59:00.555997 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.555976 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:00.556681 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556664 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 07 08:59:00.556681 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556683 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 07 08:59:00.556812 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556684 2558 factory.go:55] Registering systemd factory Sep 07 08:59:00.556812 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556702 2558 factory.go:223] Registration of the systemd container factory successfully Sep 07 08:59:00.556812 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556663 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 07 08:59:00.556812 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556782 2558 reconstruct.go:97] "Volume reconstruction finished" Sep 07 08:59:00.556812 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556793 2558 reconciler.go:29] "Reconciler: start to sync state" Sep 07 08:59:00.557063 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556932 2558 factory.go:153] Registering CRI-O factory Sep 07 08:59:00.557063 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.556945 2558 factory.go:223] Registration of the crio container factory successfully Sep 07 08:59:00.557063 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.557000 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 07 08:59:00.557063 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.557020 2558 factory.go:103] Registering Raw factory Sep 07 08:59:00.557063 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.557036 2558 manager.go:1196] Started watching for new ooms in manager Sep 07 08:59:00.557464 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.557449 2558 manager.go:349] Starting recovery of all containers Sep 07 08:59:00.561000 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.560977 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 08:59:00.564272 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.564105 2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-66.ec2.internal\" not found" node="ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.567012 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.566995 2558 manager.go:354] Recovery completed Sep 07 08:59:00.572031 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.572015 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 08:59:00.574238 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.574223 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientMemory" Sep 07 08:59:00.574322 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.574250 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 08:59:00.574322 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.574267 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientPID" Sep 07 08:59:00.574895 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.574872 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 07 08:59:00.574895 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.574886 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 07 08:59:00.575024 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.574907 2558 state_mem.go:36] "Initialized new in-memory state store" Sep 07 08:59:00.575024 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.574918 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 08:59:00.575678 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.575665 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientMemory" Sep 07 08:59:00.575743 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.575690 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 08:59:00.575743 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.575701 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientPID" Sep 07 08:59:00.577113 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.577100 2558 policy_none.go:49] "None policy: Start" Sep 07 08:59:00.577180 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.577118 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 07 08:59:00.577180 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.577131 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 07 08:59:00.579035 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.579022 2558 policy_none.go:47] "Start" Sep 07 08:59:00.601021 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.601000 2558 manager.go:337] "Starting Device Plugin manager" Sep 07 08:59:00.601122 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.601055 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 07 08:59:00.601122 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.601069 2558 server.go:87] "Starting device plugin registration server" Sep 07 08:59:00.601443 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.601431 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 07 08:59:00.601498 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.601446 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 07 08:59:00.601665 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.601645 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 07 08:59:00.601737 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.601726 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 07 08:59:00.601779 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.601739 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 07 08:59:00.602266 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.602248 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 07 08:59:00.602339 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.602305 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:00.661535 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.661499 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 07 08:59:00.662722 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.662702 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 07 08:59:00.662824 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.662761 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 07 08:59:00.662824 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.662787 2558 kubelet.go:2448] "Starting kubelet main sync loop" Sep 07 08:59:00.662925 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.662884 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 07 08:59:00.665240 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.665224 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 08:59:00.702355 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.702325 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 08:59:00.703488 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.703469 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientMemory" Sep 07 08:59:00.703625 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.703508 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 08:59:00.703625 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.703524 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientPID" Sep 07 08:59:00.703625 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.703552 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.711471 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.711399 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.711471 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.711426 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-66.ec2.internal\": node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:00.727669 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.727641 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:00.763487 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.763458 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal"] Sep 07 08:59:00.763603 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.763537 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 08:59:00.764520 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.764503 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientMemory" Sep 07 08:59:00.764596 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.764537 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 08:59:00.764596 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.764551 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientPID" Sep 07 08:59:00.766873 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.766860 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 08:59:00.767020 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.766988 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.767076 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.767022 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 08:59:00.767664 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.767646 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientMemory" Sep 07 08:59:00.767755 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.767675 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 08:59:00.767755 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.767689 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientPID" Sep 07 08:59:00.767755 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.767712 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientMemory" Sep 07 08:59:00.767755 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.767731 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 08:59:00.767755 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.767742 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientPID" Sep 07 08:59:00.769869 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.769853 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.769947 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.769881 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 07 08:59:00.770964 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.770938 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientMemory" Sep 07 08:59:00.770964 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.770966 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasNoDiskPressure" Sep 07 08:59:00.771088 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.770980 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeHasSufficientPID" Sep 07 08:59:00.787907 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.787885 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-66.ec2.internal\" not found" node="ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.791768 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.791754 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-66.ec2.internal\" not found" node="ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.828344 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.828321 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:00.859001 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.858977 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/755581d7ccdadfafac2b0c68b4d16551-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal\" (UID: \"755581d7ccdadfafac2b0c68b4d16551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.859086 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.859006 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/755581d7ccdadfafac2b0c68b4d16551-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal\" (UID: \"755581d7ccdadfafac2b0c68b4d16551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.859086 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.859024 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fba8ac2b9d64ff8f1c61c36855694155-config\") pod \"kube-apiserver-proxy-ip-10-0-131-66.ec2.internal\" (UID: \"fba8ac2b9d64ff8f1c61c36855694155\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.929380 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:00.929341 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:00.959807 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.959780 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/755581d7ccdadfafac2b0c68b4d16551-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal\" (UID: \"755581d7ccdadfafac2b0c68b4d16551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.959878 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.959804 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/755581d7ccdadfafac2b0c68b4d16551-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal\" (UID: \"755581d7ccdadfafac2b0c68b4d16551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.959878 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.959824 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/755581d7ccdadfafac2b0c68b4d16551-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal\" (UID: \"755581d7ccdadfafac2b0c68b4d16551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.959878 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.959859 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fba8ac2b9d64ff8f1c61c36855694155-config\") pod \"kube-apiserver-proxy-ip-10-0-131-66.ec2.internal\" (UID: \"fba8ac2b9d64ff8f1c61c36855694155\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.959878 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.959873 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/755581d7ccdadfafac2b0c68b4d16551-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal\" (UID: \"755581d7ccdadfafac2b0c68b4d16551\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:00.959999 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:00.959896 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fba8ac2b9d64ff8f1c61c36855694155-config\") pod \"kube-apiserver-proxy-ip-10-0-131-66.ec2.internal\" (UID: \"fba8ac2b9d64ff8f1c61c36855694155\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" Sep 07 08:59:01.030098 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:01.030024 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:01.090723 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:01.090683 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:01.094409 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:01.094378 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" Sep 07 08:59:01.131009 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:01.130978 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:01.231581 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:01.231540 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:01.332159 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:01.332079 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:02.700756 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:02.700726 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 08:59:02.715031 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:02.715003 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:02.765541 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:02.765513 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 07 08:59:02.765958 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:02.765638 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 08:59:02.765958 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:02.765670 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 08:59:02.765958 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:02.765670 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 07 08:59:02.815863 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:02.815829 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:02.837797 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:02.837764 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 08:54:01 +0000 UTC" deadline="2028-05-20 13:54:03.953923164 +0000 UTC" Sep 07 08:59:02.837797 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:02.837792 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14908h55m1.116133727s" Sep 07 08:59:02.916565 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:02.916495 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:03.014753 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:03.014694 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba8ac2b9d64ff8f1c61c36855694155.slice/crio-56ace1b8b181afeb0e6ba7ed4a1330cfb86b5f0d7a8647558f74f807afd08309 WatchSource:0}: Error finding container 56ace1b8b181afeb0e6ba7ed4a1330cfb86b5f0d7a8647558f74f807afd08309: Status 404 returned error can't find the container with id 56ace1b8b181afeb0e6ba7ed4a1330cfb86b5f0d7a8647558f74f807afd08309 Sep 07 08:59:03.015184 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:03.015164 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755581d7ccdadfafac2b0c68b4d16551.slice/crio-42cb86d80fd52f7008ae0e65b44d8b70029065759f78b95b9b14760497acc74b WatchSource:0}: Error finding container 42cb86d80fd52f7008ae0e65b44d8b70029065759f78b95b9b14760497acc74b: Status 404 returned error can't find the container with id 42cb86d80fd52f7008ae0e65b44d8b70029065759f78b95b9b14760497acc74b Sep 07 08:59:03.017366 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.017347 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:03.019655 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.019643 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 08:59:03.118280 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.118233 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-66.ec2.internal\" not found" Sep 07 08:59:03.135226 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.135195 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 08:59:03.138272 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.138258 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" Sep 07 08:59:03.148056 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.148038 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 08:59:03.149013 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.148998 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" Sep 07 08:59:03.154849 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.154833 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 07 08:59:03.726913 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.726880 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 08:59:03.815550 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.815518 2558 apiserver.go:52] "Watching apiserver" Sep 07 08:59:03.826990 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.826950 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 07 08:59:03.827390 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.827350 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-kl8g9","kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp","openshift-dns/node-resolver-mtb9n","openshift-multus/multus-wf8cn","openshift-network-operator/iptables-alerter-b2kvr","openshift-ovn-kubernetes/ovnkube-node-5gdns","kube-system/global-pull-secret-syncer-mhtrz","kube-system/konnectivity-agent-ng2zc","openshift-cluster-node-tuning-operator/tuned-msjw9","openshift-image-registry/node-ca-f75cx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal","openshift-multus/multus-additional-cni-plugins-mjtz8","openshift-multus/network-metrics-daemon-nzmrx"] Sep 07 08:59:03.829867 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.829846 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.832219 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.832195 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.832567 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.832387 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 07 08:59:03.832567 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.832513 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kssgh\"" Sep 07 08:59:03.832839 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.832653 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 07 08:59:03.834354 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.834335 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 07 08:59:03.834856 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.834593 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.834856 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.834677 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 07 08:59:03.834856 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.834696 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 07 08:59:03.835032 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.834888 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r2ppj\"" Sep 07 08:59:03.835032 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.834952 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 07 08:59:03.835151 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.835137 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 07 08:59:03.836471 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.836453 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:03.836570 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.836510 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:03.836808 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.836787 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 07 08:59:03.836894 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.836809 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 07 08:59:03.836894 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.836792 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hzjn7\"" Sep 07 08:59:03.837141 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.836791 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 07 08:59:03.839249 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.839002 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.841065 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.841048 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.841366 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.841347 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 07 08:59:03.841600 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.841579 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5plsn\"" Sep 07 08:59:03.843034 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.843015 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.843289 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.843262 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 07 08:59:03.843923 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.843903 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9m8p9\"" Sep 07 08:59:03.844010 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.843927 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 07 08:59:03.844073 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.844022 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 07 08:59:03.845117 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.845097 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 07 08:59:03.845207 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.845134 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 07 08:59:03.845364 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.845341 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:03.845464 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.845424 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:03.845868 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.845809 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 07 08:59:03.845939 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.845869 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r9b7x\"" Sep 07 08:59:03.845939 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.845814 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 07 08:59:03.846626 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.846599 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 07 08:59:03.846714 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.846691 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 07 08:59:03.847490 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.847474 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:03.848765 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.848749 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 07 08:59:03.849969 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.849949 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 07 08:59:03.849969 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.849960 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w4g7r\"" Sep 07 08:59:03.850447 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.850426 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 07 08:59:03.850558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.850540 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.851807 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.851781 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs7h2\" (UniqueName: \"kubernetes.io/projected/672d5888-106e-4eef-9a3d-0f998e395ab7-kube-api-access-zs7h2\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.851896 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.851845 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4mw8\" (UniqueName: \"kubernetes.io/projected/a55e1e5e-f140-46be-bdb5-1bc383300d9e-kube-api-access-z4mw8\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.851896 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.851888 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-sys\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852003 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.851925 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-system-cni-dir\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.852003 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.851957 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.852003 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.851993 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.852192 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e187155-3f93-4931-ba83-8f896446b9c2-cni-binary-copy\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.852192 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852141 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-kubelet\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.852192 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852174 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7fhn\" (UniqueName: \"kubernetes.io/projected/bdb82087-8753-4795-80ff-c9eae563fcc3-kube-api-access-j7fhn\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.852351 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852208 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysconfig\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852351 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852239 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysctl-conf\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852351 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852270 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-cnibin\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.852351 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852318 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-cnibin\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.852351 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852347 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-hostroot\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.852586 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852427 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0e187155-3f93-4931-ba83-8f896446b9c2-multus-daemon-config\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.852586 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/672d5888-106e-4eef-9a3d-0f998e395ab7-host-slash\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.852586 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852559 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:03.852725 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852602 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-conf-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.852725 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852654 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysctl-d\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852725 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852679 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-run\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852862 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852804 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qglp\" (UniqueName: \"kubernetes.io/projected/1c886856-6b16-4463-a537-dd6582f43f38-kube-api-access-8qglp\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852862 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852845 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55e1e5e-f140-46be-bdb5-1bc383300d9e-tmp-dir\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.852954 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-var-lib-kubelet\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852954 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852910 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-host\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852954 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852926 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c886856-6b16-4463-a537-dd6582f43f38-tmp\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.852954 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852948 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-netns\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.852954 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852950 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852968 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-cni-multus\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.852987 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/672d5888-106e-4eef-9a3d-0f998e395ab7-iptables-alerter-script\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853007 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-kubernetes\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853023 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-systemd\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853041 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-lib-modules\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853079 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853100 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-socket-dir-parent\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853144 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-multus-certs\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853189 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853167 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-modprobe-d\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853196 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-system-cni-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853222 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-cni-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853243 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-k8s-cni-cncf-io\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853260 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mxdzv\"" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853270 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-cni-bin\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853272 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853285 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1c886856-6b16-4463-a537-dd6582f43f38-etc-tuned\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853327 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a55e1e5e-f140-46be-bdb5-1bc383300d9e-hosts-file\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853375 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-etc-kubernetes\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.853615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853395 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5kqv\" (UniqueName: \"kubernetes.io/projected/0e187155-3f93-4931-ba83-8f896446b9c2-kube-api-access-n5kqv\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.854080 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.853662 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 07 08:59:03.854656 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.854601 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:03.857408 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.857385 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 07 08:59:03.857492 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.857412 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 07 08:59:03.857492 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.857418 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fbrg4\"" Sep 07 08:59:03.857492 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.857385 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 07 08:59:03.857869 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.857850 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:03.857939 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.857917 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:03.869579 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.869553 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6x99w" Sep 07 08:59:03.875500 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.875467 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6x99w" Sep 07 08:59:03.940383 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.940354 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 07 08:59:03.950215 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.950126 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" event={"ID":"755581d7ccdadfafac2b0c68b4d16551","Type":"ContainerStarted","Data":"42cb86d80fd52f7008ae0e65b44d8b70029065759f78b95b9b14760497acc74b"} Sep 07 08:59:03.951675 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.951647 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" event={"ID":"fba8ac2b9d64ff8f1c61c36855694155","Type":"ContainerStarted","Data":"56ace1b8b181afeb0e6ba7ed4a1330cfb86b5f0d7a8647558f74f807afd08309"} Sep 07 08:59:03.953988 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.953959 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.954098 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954006 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-kubelet\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.954098 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954031 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-hostroot\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.954098 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954053 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0e187155-3f93-4931-ba83-8f896446b9c2-multus-daemon-config\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.954098 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954076 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysctl-conf\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.954317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954146 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-kubelet\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.954317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954196 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-systemd-units\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.954317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954223 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysctl-conf\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.954317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954225 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-cni-netd\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.954317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954267 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-env-overrides\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954330 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954366 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-socket-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954398 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954424 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-conf-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-systemd\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954479 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-ovnkube-script-lib\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h2pq\" (UniqueName: \"kubernetes.io/projected/1369d2eb-8b75-4061-9769-b75f5e840e21-kube-api-access-5h2pq\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954543 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-serviceca\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:03.954558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954547 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-conf-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954568 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srlzv\" (UniqueName: \"kubernetes.io/projected/88c0366b-495b-4f0f-b680-64de8f3f049c-kube-api-access-srlzv\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954595 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qglp\" (UniqueName: \"kubernetes.io/projected/1c886856-6b16-4463-a537-dd6582f43f38-kube-api-access-8qglp\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954643 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-var-lib-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954686 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954734 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0e187155-3f93-4931-ba83-8f896446b9c2-multus-daemon-config\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954756 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954797 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-ovn\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954801 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-hostroot\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954821 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954846 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/035007ac-ac83-4609-8901-0a423a205428-konnectivity-ca\") pod \"konnectivity-agent-ng2zc\" (UID: \"035007ac-ac83-4609-8901-0a423a205428\") " pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954894 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1369d2eb-8b75-4061-9769-b75f5e840e21-ovn-node-metrics-cert\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954918 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/035007ac-ac83-4609-8901-0a423a205428-agent-certs\") pod \"konnectivity-agent-ng2zc\" (UID: \"035007ac-ac83-4609-8901-0a423a205428\") " pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:03.955127 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954941 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-sys-fs\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955165 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-kubernetes\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.954967 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-kubernetes\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955480 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-systemd\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955513 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-multus-certs\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955546 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-slash\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955585 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955586 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-systemd\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955627 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-modprobe-d\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955656 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-multus-certs\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955661 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-cni-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955711 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-k8s-cni-cncf-io\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955716 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-cni-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955739 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-etc-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955755 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-k8s-cni-cncf-io\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955763 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955786 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-modprobe-d\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.955802 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955798 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1c886856-6b16-4463-a537-dd6582f43f38-etc-tuned\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955851 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-etc-kubernetes\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955878 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5kqv\" (UniqueName: \"kubernetes.io/projected/0e187155-3f93-4931-ba83-8f896446b9c2-kube-api-access-n5kqv\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955905 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-ovnkube-config\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955930 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-etc-kubernetes\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955939 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955962 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e187155-3f93-4931-ba83-8f896446b9c2-cni-binary-copy\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.955986 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41bba008-7502-4a97-8759-27e21db291c1-kubelet-config\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956010 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7fhn\" (UniqueName: \"kubernetes.io/projected/bdb82087-8753-4795-80ff-c9eae563fcc3-kube-api-access-j7fhn\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956035 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysconfig\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956073 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-cnibin\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956096 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-cnibin\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956129 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/672d5888-106e-4eef-9a3d-0f998e395ab7-host-slash\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-kubelet\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956190 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-run-ovn-kubernetes\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956214 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysctl-d\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956234 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-run\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.956526 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956255 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55e1e5e-f140-46be-bdb5-1bc383300d9e-tmp-dir\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956279 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-log-socket\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956318 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-cni-bin\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956343 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4hcc\" (UniqueName: \"kubernetes.io/projected/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-kube-api-access-f4hcc\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956367 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-var-lib-kubelet\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956389 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-host\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956414 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c886856-6b16-4463-a537-dd6582f43f38-tmp\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956445 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-netns\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956470 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-cni-multus\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956462 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956496 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/672d5888-106e-4eef-9a3d-0f998e395ab7-iptables-alerter-script\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956511 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysctl-d\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956523 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-lib-modules\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956536 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-run\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956557 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956583 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-socket-dir-parent\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956608 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-node-log\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956616 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e187155-3f93-4931-ba83-8f896446b9c2-cni-binary-copy\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.957317 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956632 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-system-cni-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956655 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-cni-bin\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956681 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-var-lib-kubelet\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956682 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956704 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-etc-sysconfig\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956713 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41bba008-7502-4a97-8759-27e21db291c1-dbus\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956715 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a55e1e5e-f140-46be-bdb5-1bc383300d9e-tmp-dir\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956738 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-cni-multus\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956775 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-cnibin\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956781 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/672d5888-106e-4eef-9a3d-0f998e395ab7-host-slash\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt4k2\" (UniqueName: \"kubernetes.io/projected/3ce552e2-a957-4a53-b836-e8ffec7bfb37-kube-api-access-qt4k2\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956816 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-cnibin\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956846 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956873 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a55e1e5e-f140-46be-bdb5-1bc383300d9e-hosts-file\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956895 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs7h2\" (UniqueName: \"kubernetes.io/projected/672d5888-106e-4eef-9a3d-0f998e395ab7-kube-api-access-zs7h2\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956898 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-var-lib-cni-bin\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956918 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-run-netns\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:03.958183 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956940 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-host-run-netns\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956981 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-host\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956974 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957013 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-registration-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957032 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-multus-socket-dir-parent\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957039 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-device-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957071 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4mw8\" (UniqueName: \"kubernetes.io/projected/a55e1e5e-f140-46be-bdb5-1bc383300d9e-kube-api-access-z4mw8\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957084 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-lib-modules\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957099 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-sys\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957073 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e187155-3f93-4931-ba83-8f896446b9c2-system-cni-dir\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957134 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a55e1e5e-f140-46be-bdb5-1bc383300d9e-hosts-file\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957171 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-system-cni-dir\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957192 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/672d5888-106e-4eef-9a3d-0f998e395ab7-iptables-alerter-script\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.956489 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957236 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-host\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957268 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bdb82087-8753-4795-80ff-c9eae563fcc3-system-cni-dir\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.957419 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c886856-6b16-4463-a537-dd6582f43f38-sys\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.958984 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.958100 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bdb82087-8753-4795-80ff-c9eae563fcc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.959975 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.959942 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 08:59:03.960072 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.959980 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 08:59:03.960072 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.959995 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5g66j for pod openshift-network-diagnostics/network-check-target-kl8g9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:03.960072 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:03.960067 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j podName:854c6727-7f69-4d5f-bf0c-ade9ed39908a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:04.46004696 +0000 UTC m=+2.976837859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5g66j" (UniqueName: "kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j") pod "network-check-target-kl8g9" (UID: "854c6727-7f69-4d5f-bf0c-ade9ed39908a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:03.961362 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.961088 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c886856-6b16-4463-a537-dd6582f43f38-tmp\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.963825 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.963757 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1c886856-6b16-4463-a537-dd6582f43f38-etc-tuned\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.966281 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.966030 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5kqv\" (UniqueName: \"kubernetes.io/projected/0e187155-3f93-4931-ba83-8f896446b9c2-kube-api-access-n5kqv\") pod \"multus-wf8cn\" (UID: \"0e187155-3f93-4931-ba83-8f896446b9c2\") " pod="openshift-multus/multus-wf8cn" Sep 07 08:59:03.966281 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.966048 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qglp\" (UniqueName: \"kubernetes.io/projected/1c886856-6b16-4463-a537-dd6582f43f38-kube-api-access-8qglp\") pod \"tuned-msjw9\" (UID: \"1c886856-6b16-4463-a537-dd6582f43f38\") " pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:03.966580 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.966547 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7fhn\" (UniqueName: \"kubernetes.io/projected/bdb82087-8753-4795-80ff-c9eae563fcc3-kube-api-access-j7fhn\") pod \"multus-additional-cni-plugins-mjtz8\" (UID: \"bdb82087-8753-4795-80ff-c9eae563fcc3\") " pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:03.967698 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.967648 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4mw8\" (UniqueName: \"kubernetes.io/projected/a55e1e5e-f140-46be-bdb5-1bc383300d9e-kube-api-access-z4mw8\") pod \"node-resolver-mtb9n\" (UID: \"a55e1e5e-f140-46be-bdb5-1bc383300d9e\") " pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:03.967793 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:03.967714 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs7h2\" (UniqueName: \"kubernetes.io/projected/672d5888-106e-4eef-9a3d-0f998e395ab7-kube-api-access-zs7h2\") pod \"iptables-alerter-b2kvr\" (UID: \"672d5888-106e-4eef-9a3d-0f998e395ab7\") " pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:04.057787 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057704 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41bba008-7502-4a97-8759-27e21db291c1-kubelet-config\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:04.057787 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057742 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-kubelet\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.057787 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057757 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-run-ovn-kubernetes\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.057787 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057782 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-log-socket\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057804 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-cni-bin\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4hcc\" (UniqueName: \"kubernetes.io/projected/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-kube-api-access-f4hcc\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057847 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/41bba008-7502-4a97-8759-27e21db291c1-kubelet-config\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057856 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-node-log\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057881 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-log-socket\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057893 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057899 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-run-ovn-kubernetes\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057888 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-node-log\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057855 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-kubelet\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057900 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-cni-bin\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057934 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.057937 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41bba008-7502-4a97-8759-27e21db291c1-dbus\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:04.058096 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058004 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt4k2\" (UniqueName: \"kubernetes.io/projected/3ce552e2-a957-4a53-b836-e8ffec7bfb37-kube-api-access-qt4k2\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058131 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/41bba008-7502-4a97-8759-27e21db291c1-dbus\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058154 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-run-netns\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058172 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-host\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058187 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-registration-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058199 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-run-netns\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058209 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-device-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058231 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-host\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-systemd-units\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058265 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-registration-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058272 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-systemd-units\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-cni-netd\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058315 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-cni-netd\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058323 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-device-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058330 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-env-overrides\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058379 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058403 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-socket-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.058761 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058452 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058457 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-systemd\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058489 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-ovnkube-script-lib\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058499 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-systemd\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h2pq\" (UniqueName: \"kubernetes.io/projected/1369d2eb-8b75-4061-9769-b75f5e840e21-kube-api-access-5h2pq\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058541 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-serviceca\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058564 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srlzv\" (UniqueName: \"kubernetes.io/projected/88c0366b-495b-4f0f-b680-64de8f3f049c-kube-api-access-srlzv\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058585 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-var-lib-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058599 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058613 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-ovn\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058612 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-socket-dir\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.058697 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.058754 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret podName:41bba008-7502-4a97-8759-27e21db291c1 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:04.558738454 +0000 UTC m=+3.075529336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret") pod "global-pull-secret-syncer-mhtrz" (UID: "41bba008-7502-4a97-8759-27e21db291c1") : object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058778 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058811 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-run-ovn\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058821 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-env-overrides\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.059592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058826 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-var-lib-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058855 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/035007ac-ac83-4609-8901-0a423a205428-konnectivity-ca\") pod \"konnectivity-agent-ng2zc\" (UID: \"035007ac-ac83-4609-8901-0a423a205428\") " pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058886 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1369d2eb-8b75-4061-9769-b75f5e840e21-ovn-node-metrics-cert\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/035007ac-ac83-4609-8901-0a423a205428-agent-certs\") pod \"konnectivity-agent-ng2zc\" (UID: \"035007ac-ac83-4609-8901-0a423a205428\") " pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-sys-fs\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058960 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-slash\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058983 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-ovnkube-script-lib\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.058995 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059028 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-etc-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059046 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-sys-fs\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059062 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059078 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-host-slash\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059090 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-ovnkube-config\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.059107 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059116 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1369d2eb-8b75-4061-9769-b75f5e840e21-etc-openvswitch\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.059173 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs podName:3ce552e2-a957-4a53-b836-e8ffec7bfb37 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:04.559155011 +0000 UTC m=+3.075945906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs") pod "network-metrics-daemon-nzmrx" (UID: "3ce552e2-a957-4a53-b836-e8ffec7bfb37") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059226 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/88c0366b-495b-4f0f-b680-64de8f3f049c-etc-selinux\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.060177 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059324 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-serviceca\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:04.060760 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059473 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/035007ac-ac83-4609-8901-0a423a205428-konnectivity-ca\") pod \"konnectivity-agent-ng2zc\" (UID: \"035007ac-ac83-4609-8901-0a423a205428\") " pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:04.060760 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.059588 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1369d2eb-8b75-4061-9769-b75f5e840e21-ovnkube-config\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.061527 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.061503 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/035007ac-ac83-4609-8901-0a423a205428-agent-certs\") pod \"konnectivity-agent-ng2zc\" (UID: \"035007ac-ac83-4609-8901-0a423a205428\") " pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:04.061641 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.061623 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1369d2eb-8b75-4061-9769-b75f5e840e21-ovn-node-metrics-cert\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.067278 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.067254 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt4k2\" (UniqueName: \"kubernetes.io/projected/3ce552e2-a957-4a53-b836-e8ffec7bfb37-kube-api-access-qt4k2\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:04.067445 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.067285 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srlzv\" (UniqueName: \"kubernetes.io/projected/88c0366b-495b-4f0f-b680-64de8f3f049c-kube-api-access-srlzv\") pod \"aws-ebs-csi-driver-node-nd8dp\" (UID: \"88c0366b-495b-4f0f-b680-64de8f3f049c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.067572 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.067539 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h2pq\" (UniqueName: \"kubernetes.io/projected/1369d2eb-8b75-4061-9769-b75f5e840e21-kube-api-access-5h2pq\") pod \"ovnkube-node-5gdns\" (UID: \"1369d2eb-8b75-4061-9769-b75f5e840e21\") " pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.068252 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.068229 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4hcc\" (UniqueName: \"kubernetes.io/projected/0cdd0beb-99bd-4921-ba51-e135d1c0a6db-kube-api-access-f4hcc\") pod \"node-ca-f75cx\" (UID: \"0cdd0beb-99bd-4921-ba51-e135d1c0a6db\") " pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:04.143528 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.143493 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-msjw9" Sep 07 08:59:04.152737 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.152709 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtb9n" Sep 07 08:59:04.163316 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.163284 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" Sep 07 08:59:04.169026 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.168998 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wf8cn" Sep 07 08:59:04.176685 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.176664 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b2kvr" Sep 07 08:59:04.184347 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.184322 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:04.192053 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.192035 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:04.198843 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.198544 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" Sep 07 08:59:04.203754 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.203726 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f75cx" Sep 07 08:59:04.292095 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.292067 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 08:59:04.461179 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.461090 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:04.461332 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.461266 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 08:59:04.461332 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.461307 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 08:59:04.461332 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.461322 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5g66j for pod openshift-network-diagnostics/network-check-target-kl8g9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:04.461456 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.461382 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j podName:854c6727-7f69-4d5f-bf0c-ade9ed39908a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:05.461367844 +0000 UTC m=+3.978158725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5g66j" (UniqueName: "kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j") pod "network-check-target-kl8g9" (UID: "854c6727-7f69-4d5f-bf0c-ade9ed39908a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:04.562263 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.562231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:04.562415 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.562278 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:04.562415 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.562392 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:04.562510 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.562459 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret podName:41bba008-7502-4a97-8759-27e21db291c1 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:05.562444167 +0000 UTC m=+4.079235047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret") pod "global-pull-secret-syncer-mhtrz" (UID: "41bba008-7502-4a97-8759-27e21db291c1") : object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:04.562510 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.562396 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:04.562591 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:04.562543 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs podName:3ce552e2-a957-4a53-b836-e8ffec7bfb37 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:05.562506241 +0000 UTC m=+4.079297123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs") pod "network-metrics-daemon-nzmrx" (UID: "3ce552e2-a957-4a53-b836-e8ffec7bfb37") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:04.873591 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.873560 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb82087_8753_4795_80ff_c9eae563fcc3.slice/crio-bebdec6708e4fa81e8fae708d7fc93f0388bc1d2c10995f28d433fa60a0784e6 WatchSource:0}: Error finding container bebdec6708e4fa81e8fae708d7fc93f0388bc1d2c10995f28d433fa60a0784e6: Status 404 returned error can't find the container with id bebdec6708e4fa81e8fae708d7fc93f0388bc1d2c10995f28d433fa60a0784e6 Sep 07 08:59:04.878529 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.878507 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1369d2eb_8b75_4061_9769_b75f5e840e21.slice/crio-2c560d13e3f4b56717ab5d7453eb60dd1f6663dfaa6de8f44d285c0f65bbfce5 WatchSource:0}: Error finding container 2c560d13e3f4b56717ab5d7453eb60dd1f6663dfaa6de8f44d285c0f65bbfce5: Status 404 returned error can't find the container with id 2c560d13e3f4b56717ab5d7453eb60dd1f6663dfaa6de8f44d285c0f65bbfce5 Sep 07 08:59:04.878603 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.878528 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 08:54:03 +0000 UTC" deadline="2028-04-04 11:31:02.961664727 +0000 UTC" Sep 07 08:59:04.878603 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.878556 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13802h31m58.083112981s" Sep 07 08:59:04.879327 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.879288 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c886856_6b16_4463_a537_dd6582f43f38.slice/crio-aed39a4a3418f7730e01ad63663fe72628057467dc793011e98ee5b4d213de07 WatchSource:0}: Error finding container aed39a4a3418f7730e01ad63663fe72628057467dc793011e98ee5b4d213de07: Status 404 returned error can't find the container with id aed39a4a3418f7730e01ad63663fe72628057467dc793011e98ee5b4d213de07 Sep 07 08:59:04.880678 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.880657 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e187155_3f93_4931_ba83_8f896446b9c2.slice/crio-cb550cbc40ef63dfa24305f237c5b52871069977f4edfcbaa0dd6383cbd2fbef WatchSource:0}: Error finding container cb550cbc40ef63dfa24305f237c5b52871069977f4edfcbaa0dd6383cbd2fbef: Status 404 returned error can't find the container with id cb550cbc40ef63dfa24305f237c5b52871069977f4edfcbaa0dd6383cbd2fbef Sep 07 08:59:04.881611 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.881589 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda55e1e5e_f140_46be_bdb5_1bc383300d9e.slice/crio-dbc2398abdd6447020a969dc0d1b01a926145fdc2311b8aac612f9f0452306c8 WatchSource:0}: Error finding container dbc2398abdd6447020a969dc0d1b01a926145fdc2311b8aac612f9f0452306c8: Status 404 returned error can't find the container with id dbc2398abdd6447020a969dc0d1b01a926145fdc2311b8aac612f9f0452306c8 Sep 07 08:59:04.882875 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.882836 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035007ac_ac83_4609_8901_0a423a205428.slice/crio-a4bc905d472c4234bedb990a9370bc91dbf4b8767264d5c4047451c608eeb44e WatchSource:0}: Error finding container a4bc905d472c4234bedb990a9370bc91dbf4b8767264d5c4047451c608eeb44e: Status 404 returned error can't find the container with id a4bc905d472c4234bedb990a9370bc91dbf4b8767264d5c4047451c608eeb44e Sep 07 08:59:04.883670 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.883643 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672d5888_106e_4eef_9a3d_0f998e395ab7.slice/crio-7722c67601b9c8096f9d7395d0a3f6c53f6da180967eda4fb0c44c6fec456add WatchSource:0}: Error finding container 7722c67601b9c8096f9d7395d0a3f6c53f6da180967eda4fb0c44c6fec456add: Status 404 returned error can't find the container with id 7722c67601b9c8096f9d7395d0a3f6c53f6da180967eda4fb0c44c6fec456add Sep 07 08:59:04.884449 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.884428 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c0366b_495b_4f0f_b680_64de8f3f049c.slice/crio-b38c203d39f77498d4392305713e32e6ce520dbde66fb7ebe47c41562f1584df WatchSource:0}: Error finding container b38c203d39f77498d4392305713e32e6ce520dbde66fb7ebe47c41562f1584df: Status 404 returned error can't find the container with id b38c203d39f77498d4392305713e32e6ce520dbde66fb7ebe47c41562f1584df Sep 07 08:59:04.907356 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:04.907326 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cdd0beb_99bd_4921_ba51_e135d1c0a6db.slice/crio-5a75674c3541e6eb0fa6a7ac573138e1a25679f5bf948f7a5e47c814903ada92 WatchSource:0}: Error finding container 5a75674c3541e6eb0fa6a7ac573138e1a25679f5bf948f7a5e47c814903ada92: Status 404 returned error can't find the container with id 5a75674c3541e6eb0fa6a7ac573138e1a25679f5bf948f7a5e47c814903ada92 Sep 07 08:59:04.958662 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.958631 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f75cx" event={"ID":"0cdd0beb-99bd-4921-ba51-e135d1c0a6db","Type":"ContainerStarted","Data":"5a75674c3541e6eb0fa6a7ac573138e1a25679f5bf948f7a5e47c814903ada92"} Sep 07 08:59:04.959858 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.959831 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" event={"ID":"88c0366b-495b-4f0f-b680-64de8f3f049c","Type":"ContainerStarted","Data":"b38c203d39f77498d4392305713e32e6ce520dbde66fb7ebe47c41562f1584df"} Sep 07 08:59:04.961073 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.961051 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2kvr" event={"ID":"672d5888-106e-4eef-9a3d-0f998e395ab7","Type":"ContainerStarted","Data":"7722c67601b9c8096f9d7395d0a3f6c53f6da180967eda4fb0c44c6fec456add"} Sep 07 08:59:04.962190 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.962163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ng2zc" event={"ID":"035007ac-ac83-4609-8901-0a423a205428","Type":"ContainerStarted","Data":"a4bc905d472c4234bedb990a9370bc91dbf4b8767264d5c4047451c608eeb44e"} Sep 07 08:59:04.963485 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.963459 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtb9n" event={"ID":"a55e1e5e-f140-46be-bdb5-1bc383300d9e","Type":"ContainerStarted","Data":"dbc2398abdd6447020a969dc0d1b01a926145fdc2311b8aac612f9f0452306c8"} Sep 07 08:59:04.964695 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.964671 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wf8cn" event={"ID":"0e187155-3f93-4931-ba83-8f896446b9c2","Type":"ContainerStarted","Data":"cb550cbc40ef63dfa24305f237c5b52871069977f4edfcbaa0dd6383cbd2fbef"} Sep 07 08:59:04.965979 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.965949 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-msjw9" event={"ID":"1c886856-6b16-4463-a537-dd6582f43f38","Type":"ContainerStarted","Data":"aed39a4a3418f7730e01ad63663fe72628057467dc793011e98ee5b4d213de07"} Sep 07 08:59:04.967284 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.967260 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"2c560d13e3f4b56717ab5d7453eb60dd1f6663dfaa6de8f44d285c0f65bbfce5"} Sep 07 08:59:04.968346 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:04.968328 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerStarted","Data":"bebdec6708e4fa81e8fae708d7fc93f0388bc1d2c10995f28d433fa60a0784e6"} Sep 07 08:59:05.406851 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.406759 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 07 08:59:05.467147 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.467109 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:05.467359 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.467339 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 08:59:05.467428 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.467367 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 08:59:05.467428 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.467380 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5g66j for pod openshift-network-diagnostics/network-check-target-kl8g9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:05.467535 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.467440 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j podName:854c6727-7f69-4d5f-bf0c-ade9ed39908a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:07.467421017 +0000 UTC m=+5.984211917 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5g66j" (UniqueName: "kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j") pod "network-check-target-kl8g9" (UID: "854c6727-7f69-4d5f-bf0c-ade9ed39908a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:05.567930 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.567897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:05.568106 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.567950 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:05.568165 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.568117 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:05.568236 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.568175 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs podName:3ce552e2-a957-4a53-b836-e8ffec7bfb37 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:07.568157105 +0000 UTC m=+6.084948000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs") pod "network-metrics-daemon-nzmrx" (UID: "3ce552e2-a957-4a53-b836-e8ffec7bfb37") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:05.568619 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.568443 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:05.568619 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.568517 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret podName:41bba008-7502-4a97-8759-27e21db291c1 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:07.568499137 +0000 UTC m=+6.085290033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret") pod "global-pull-secret-syncer-mhtrz" (UID: "41bba008-7502-4a97-8759-27e21db291c1") : object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:05.879123 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.879004 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 08:54:03 +0000 UTC" deadline="2028-05-30 14:13:53.989729162 +0000 UTC" Sep 07 08:59:05.879123 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.879062 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15149h14m48.110671023s" Sep 07 08:59:05.948201 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.948170 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:05.948390 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.948331 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:05.948729 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.948709 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:05.948818 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.948800 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:05.948876 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.948862 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:05.948977 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:05.948929 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:05.985388 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.985352 2558 generic.go:356] "Generic (PLEG): container finished" podID="755581d7ccdadfafac2b0c68b4d16551" containerID="f9392d891e041b9d780d6469c8d86a310148f7f70c6aee92898770f90f15d11c" exitCode=0 Sep 07 08:59:05.985777 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:05.985707 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" event={"ID":"755581d7ccdadfafac2b0c68b4d16551","Type":"ContainerDied","Data":"f9392d891e041b9d780d6469c8d86a310148f7f70c6aee92898770f90f15d11c"} Sep 07 08:59:06.006008 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:06.005967 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" event={"ID":"fba8ac2b9d64ff8f1c61c36855694155","Type":"ContainerStarted","Data":"b2db586b9c161f662d797952fdd65783a812351c3497b82322875a4cce3f6d18"} Sep 07 08:59:07.019642 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.019492 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" event={"ID":"755581d7ccdadfafac2b0c68b4d16551","Type":"ContainerStarted","Data":"f73cf9ccf5eca0386f994c6cd1f9cb2337060135e60db6084072f75d84b0dd7e"} Sep 07 08:59:07.034838 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.033910 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-66.ec2.internal" podStartSLOduration=4.03388521 podStartE2EDuration="4.03388521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:06.021822332 +0000 UTC m=+4.538613236" watchObservedRunningTime="2026-09-07 08:59:07.03388521 +0000 UTC m=+5.550676114" Sep 07 08:59:07.486328 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.485896 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:07.486328 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.486102 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 08:59:07.486328 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.486120 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 08:59:07.486328 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.486132 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5g66j for pod openshift-network-diagnostics/network-check-target-kl8g9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:07.486328 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.486189 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j podName:854c6727-7f69-4d5f-bf0c-ade9ed39908a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:11.48617098 +0000 UTC m=+10.002961864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5g66j" (UniqueName: "kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j") pod "network-check-target-kl8g9" (UID: "854c6727-7f69-4d5f-bf0c-ade9ed39908a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:07.587612 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.587570 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:07.587781 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.587629 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:07.587781 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.587742 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:07.587904 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.587811 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs podName:3ce552e2-a957-4a53-b836-e8ffec7bfb37 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:11.587791781 +0000 UTC m=+10.104582663 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs") pod "network-metrics-daemon-nzmrx" (UID: "3ce552e2-a957-4a53-b836-e8ffec7bfb37") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:07.587904 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.587879 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:07.588006 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.587930 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret podName:41bba008-7502-4a97-8759-27e21db291c1 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:11.587918964 +0000 UTC m=+10.104709859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret") pod "global-pull-secret-syncer-mhtrz" (UID: "41bba008-7502-4a97-8759-27e21db291c1") : object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:07.946489 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.946406 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:07.946651 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.946547 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:07.946923 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.946903 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:07.947010 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.946993 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:07.949269 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:07.949244 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:07.949445 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:07.949425 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:09.945818 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:09.945778 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:09.945818 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:09.945819 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:09.946267 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:09.945778 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:09.946267 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:09.945919 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:09.946267 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:09.946049 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:09.946267 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:09.946132 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:11.522673 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:11.522605 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:11.523110 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.522775 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 08:59:11.523110 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.522798 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 08:59:11.523110 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.522811 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5g66j for pod openshift-network-diagnostics/network-check-target-kl8g9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:11.523110 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.522879 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j podName:854c6727-7f69-4d5f-bf0c-ade9ed39908a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.522858001 +0000 UTC m=+18.039648900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5g66j" (UniqueName: "kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j") pod "network-check-target-kl8g9" (UID: "854c6727-7f69-4d5f-bf0c-ade9ed39908a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:11.623521 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:11.623484 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:11.623696 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:11.623539 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:11.623696 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.623648 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:11.623829 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.623721 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret podName:41bba008-7502-4a97-8759-27e21db291c1 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.623700718 +0000 UTC m=+18.140491613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret") pod "global-pull-secret-syncer-mhtrz" (UID: "41bba008-7502-4a97-8759-27e21db291c1") : object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:11.624014 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.623648 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:11.624014 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.623984 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs podName:3ce552e2-a957-4a53-b836-e8ffec7bfb37 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.623962339 +0000 UTC m=+18.140753235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs") pod "network-metrics-daemon-nzmrx" (UID: "3ce552e2-a957-4a53-b836-e8ffec7bfb37") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:11.946266 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:11.946194 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:11.946443 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.946291 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:11.946443 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:11.946359 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:11.946562 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.946463 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:11.946562 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:11.946500 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:11.946659 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:11.946607 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:13.946679 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:13.946480 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:13.946679 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:13.946608 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:13.946679 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:13.946480 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:13.946679 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:13.946643 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:13.947362 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:13.946711 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:13.947362 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:13.946792 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:15.944937 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:15.944855 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:15.945393 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:15.944994 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:15.945393 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:15.945078 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:15.945393 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:15.945175 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:15.945393 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:15.945228 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:15.945393 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:15.945329 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:17.945663 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:17.945564 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:17.945663 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:17.945615 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:17.946158 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:17.945577 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:17.946158 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:17.945700 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:17.946158 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:17.945800 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:17.946158 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:17.945888 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:19.582760 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:19.582676 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:19.583225 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.582881 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 08:59:19.583225 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.582907 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 08:59:19.583225 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.582920 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5g66j for pod openshift-network-diagnostics/network-check-target-kl8g9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:19.583225 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.582994 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j podName:854c6727-7f69-4d5f-bf0c-ade9ed39908a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:35.582972905 +0000 UTC m=+34.099763796 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5g66j" (UniqueName: "kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j") pod "network-check-target-kl8g9" (UID: "854c6727-7f69-4d5f-bf0c-ade9ed39908a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:19.683224 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:19.683169 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:19.683224 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:19.683228 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:19.683460 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.683337 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:19.683460 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.683410 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret podName:41bba008-7502-4a97-8759-27e21db291c1 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:35.683391314 +0000 UTC m=+34.200182199 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret") pod "global-pull-secret-syncer-mhtrz" (UID: "41bba008-7502-4a97-8759-27e21db291c1") : object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:19.683460 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.683343 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:19.683621 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.683466 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs podName:3ce552e2-a957-4a53-b836-e8ffec7bfb37 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:35.683452931 +0000 UTC m=+34.200243812 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs") pod "network-metrics-daemon-nzmrx" (UID: "3ce552e2-a957-4a53-b836-e8ffec7bfb37") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:19.944947 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:19.944867 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:19.944947 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:19.944879 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:19.945155 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:19.944874 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:19.945155 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.944995 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:19.945155 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.945121 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:19.945278 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:19.945210 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:21.946469 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:21.946435 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:21.946956 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:21.946547 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:21.946956 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:21.946657 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:21.946956 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:21.946798 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:21.946956 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:21.946834 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:21.946956 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:21.946935 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:23.053208 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.053178 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-msjw9" event={"ID":"1c886856-6b16-4463-a537-dd6582f43f38","Type":"ContainerStarted","Data":"602bfa574ae2e4df70e73540fda5c0fa3eb66434677591a6c26787564f61c5a5"} Sep 07 08:59:23.057858 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.057752 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"ce2b6c8cccc84fead6a9b738189e0acd0387ecd3a8dc0114140f2cafc1f8b293"} Sep 07 08:59:23.060034 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.059982 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerStarted","Data":"044a5f7598106cb86e6ac7ab3ad27c004821ab2c90bc3ab694487638e72dd439"} Sep 07 08:59:23.062022 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.061997 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f75cx" event={"ID":"0cdd0beb-99bd-4921-ba51-e135d1c0a6db","Type":"ContainerStarted","Data":"b747c9243d7a1cbf54133fc185e61fd892be699fd121685297bc98ac6292b40a"} Sep 07 08:59:23.064020 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.064002 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" event={"ID":"88c0366b-495b-4f0f-b680-64de8f3f049c","Type":"ContainerStarted","Data":"cff2a1c14bbe783e28d4b6eebd85f2367936f45a05efce48d45417af37fff358"} Sep 07 08:59:23.065349 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.065328 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ng2zc" event={"ID":"035007ac-ac83-4609-8901-0a423a205428","Type":"ContainerStarted","Data":"36b7fcb0c4597cc98ae79c7287ad41be11966e32b4c38a4baf666b667c891b00"} Sep 07 08:59:23.066799 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.066779 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtb9n" event={"ID":"a55e1e5e-f140-46be-bdb5-1bc383300d9e","Type":"ContainerStarted","Data":"58fa8a832f040fbd1951a927b5b198c94dbe9aeb5579e35712e13a48c3630635"} Sep 07 08:59:23.071897 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.071861 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-msjw9" podStartSLOduration=4.165905449 podStartE2EDuration="22.071850413s" totalImagesPullingTime="17.905944964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:23.071761312 +0000 UTC m=+21.588552277" watchObservedRunningTime="2026-09-07 08:59:23.071850413 +0000 UTC m=+21.588641317" Sep 07 08:59:23.071994 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.071976 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-66.ec2.internal" podStartSLOduration=20.071972574 podStartE2EDuration="20.071972574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:07.034801428 +0000 UTC m=+5.551592332" watchObservedRunningTime="2026-09-07 08:59:23.071972574 +0000 UTC m=+21.588763480" Sep 07 08:59:23.085114 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.085070 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ng2zc" podStartSLOduration=6.885976164 podStartE2EDuration="21.085054509s" totalImagesPullingTime="14.199078345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:23.084642172 +0000 UTC m=+21.601433076" watchObservedRunningTime="2026-09-07 08:59:23.085054509 +0000 UTC m=+21.601845429" Sep 07 08:59:23.097911 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.097879 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mtb9n" podStartSLOduration=3.195610478 podStartE2EDuration="21.097866968s" totalImagesPullingTime="17.90225649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:23.097644643 +0000 UTC m=+21.614435547" watchObservedRunningTime="2026-09-07 08:59:23.097866968 +0000 UTC m=+21.614657870" Sep 07 08:59:23.112192 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.112151 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f75cx" podStartSLOduration=3.2356799 podStartE2EDuration="21.112136155s" totalImagesPullingTime="17.876456255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:23.112070883 +0000 UTC m=+21.628861786" watchObservedRunningTime="2026-09-07 08:59:23.112136155 +0000 UTC m=+21.628927073" Sep 07 08:59:23.945335 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.945291 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:23.945522 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.945288 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:23.945522 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:23.945407 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:23.945522 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:23.945423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:23.945522 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:23.945505 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:23.945732 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:23.945545 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:24.070588 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.070510 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wf8cn" event={"ID":"0e187155-3f93-4931-ba83-8f896446b9c2","Type":"ContainerStarted","Data":"ed64dbe4fa6f4845a0b89d59561ded5dfa521ad2e8f590aacd8539ee34dafbc3"} Sep 07 08:59:24.073481 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.073460 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 08:59:24.073821 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.073799 2558 generic.go:356] "Generic (PLEG): container finished" podID="1369d2eb-8b75-4061-9769-b75f5e840e21" containerID="39c60e443078a3fbba7d2c7029933c2e101f5b611d06e467d15446a578e4c3f8" exitCode=1 Sep 07 08:59:24.073891 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.073872 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"aedfeba9a68afad052136bd5976faf9c8727da0494aef48bff74b054aedba755"} Sep 07 08:59:24.073969 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.073952 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"43efd794c4f6a0c73d68a4cd96254b432c6a3282f2818a1ade280ca27a44ef7f"} Sep 07 08:59:24.074032 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.074006 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"1b33fae242e47b2a32d5804354850ef7f19ea247f916c2507a39b237e03fedcd"} Sep 07 08:59:24.074090 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.074068 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"72a26b487ed1ec9069d5f80baba7df23c33cb891bc8b798d9190df9b05b718ac"} Sep 07 08:59:24.074143 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.074123 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerDied","Data":"39c60e443078a3fbba7d2c7029933c2e101f5b611d06e467d15446a578e4c3f8"} Sep 07 08:59:24.075325 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.075287 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdb82087-8753-4795-80ff-c9eae563fcc3" containerID="044a5f7598106cb86e6ac7ab3ad27c004821ab2c90bc3ab694487638e72dd439" exitCode=0 Sep 07 08:59:24.075428 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.075326 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerDied","Data":"044a5f7598106cb86e6ac7ab3ad27c004821ab2c90bc3ab694487638e72dd439"} Sep 07 08:59:24.105460 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.105420 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wf8cn" podStartSLOduration=3.905727642 podStartE2EDuration="22.105406508s" totalImagesPullingTime="18.199678866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:24.086399566 +0000 UTC m=+22.603190469" watchObservedRunningTime="2026-09-07 08:59:24.105406508 +0000 UTC m=+22.622197411" Sep 07 08:59:24.227019 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.226780 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 07 08:59:24.896644 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.896500 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T08:59:24.227016336Z","UUID":"02cac8bc-3f3c-4fd0-b3b7-14b3d29efe16","Handler":null,"Name":"","Endpoint":""} Sep 07 08:59:24.898218 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.898200 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 07 08:59:24.898353 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:24.898231 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 07 08:59:25.080784 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:25.080728 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" event={"ID":"88c0366b-495b-4f0f-b680-64de8f3f049c","Type":"ContainerStarted","Data":"594e8a2b433607335ca013e53376b1485e813ac0ae0ea62b4f610cd49aba98a7"} Sep 07 08:59:25.083228 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:25.083196 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b2kvr" event={"ID":"672d5888-106e-4eef-9a3d-0f998e395ab7","Type":"ContainerStarted","Data":"46f12ab082c9768d3c40b998f6b788257a21876192e4c869373db1e4f25ae6e8"} Sep 07 08:59:25.098452 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:25.098406 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b2kvr" podStartSLOduration=5.218872362 podStartE2EDuration="23.09838931s" totalImagesPullingTime="17.879516948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:25.098284137 +0000 UTC m=+23.615075040" watchObservedRunningTime="2026-09-07 08:59:25.09838931 +0000 UTC m=+23.615180216" Sep 07 08:59:25.945604 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:25.945531 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:25.945759 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:25.945531 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:25.945759 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:25.945656 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:25.945759 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:25.945666 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:25.945918 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:25.945757 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:25.945918 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:25.945873 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:26.086853 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:26.086818 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" event={"ID":"88c0366b-495b-4f0f-b680-64de8f3f049c","Type":"ContainerStarted","Data":"0bdf37500d70a1c48cef2a1073b19404cdccbb0bebe01482d8ba3bd58fba5668"} Sep 07 08:59:26.089797 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:26.089777 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 08:59:26.092976 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:26.090627 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"cca2a9a389b684f9efe6ff7f558337aa9584ba349d2a8b8d90c8071fc9b9e966"} Sep 07 08:59:27.243749 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:27.243715 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:27.244321 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:27.244284 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:27.262269 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:27.262219 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nd8dp" podStartSLOduration=5.293871634 podStartE2EDuration="25.262199118s" totalImagesPullingTime="19.968327484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:26.10417881 +0000 UTC m=+24.620969713" watchObservedRunningTime="2026-09-07 08:59:27.262199118 +0000 UTC m=+25.778990023" Sep 07 08:59:27.945495 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:27.945423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:27.945646 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:27.945423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:27.945646 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:27.945563 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:27.945646 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:27.945630 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:27.945806 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:27.945423 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:27.945806 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:27.945716 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:28.094968 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:28.094933 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:28.095507 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:28.095481 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ng2zc" Sep 07 08:59:28.461344 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:28.461319 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtb9n_a55e1e5e-f140-46be-bdb5-1bc383300d9e/dns-node-resolver/0.log" Sep 07 08:59:29.100435 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.100213 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 08:59:29.100883 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.100855 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"a522c16ac7cda7e02ccb8bcb775c9217e0dd5c9f6838454903825d3eb0c9a3fe"} Sep 07 08:59:29.101214 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.101196 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:29.101284 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.101269 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:29.101397 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.101382 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:29.101478 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.101456 2558 scope.go:117] "RemoveContainer" containerID="39c60e443078a3fbba7d2c7029933c2e101f5b611d06e467d15446a578e4c3f8" Sep 07 08:59:29.103062 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.103036 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerStarted","Data":"f1424224b0361834e2dcdf37bd8c34d4bf3ca7d799b1ec3e8b4644c1c2ebf52c"} Sep 07 08:59:29.117194 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.117172 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:29.117431 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.117412 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 08:59:29.640833 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.640799 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f75cx_0cdd0beb-99bd-4921-ba51-e135d1c0a6db/node-ca/0.log" Sep 07 08:59:29.945844 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.945752 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:29.945844 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.945798 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:29.946038 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:29.945769 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:29.946038 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:29.945879 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:29.946038 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:29.945969 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:29.946181 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:29.946094 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:30.108460 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.108435 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 08:59:30.108795 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.108775 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" event={"ID":"1369d2eb-8b75-4061-9769-b75f5e840e21","Type":"ContainerStarted","Data":"7dad125ab19312232514e6bb30407dc10aeefe1c3fa581dc576fb7539f7646ea"} Sep 07 08:59:30.110568 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.110542 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdb82087-8753-4795-80ff-c9eae563fcc3" containerID="f1424224b0361834e2dcdf37bd8c34d4bf3ca7d799b1ec3e8b4644c1c2ebf52c" exitCode=0 Sep 07 08:59:30.110678 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.110629 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerDied","Data":"f1424224b0361834e2dcdf37bd8c34d4bf3ca7d799b1ec3e8b4644c1c2ebf52c"} Sep 07 08:59:30.138206 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.138156 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" podStartSLOduration=10.199600435 podStartE2EDuration="28.138139493s" totalImagesPullingTime="17.938539058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:30.136032632 +0000 UTC m=+28.652823592" watchObservedRunningTime="2026-09-07 08:59:30.138139493 +0000 UTC m=+28.654930395" Sep 07 08:59:30.628799 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.628767 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mhtrz"] Sep 07 08:59:30.628948 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.628874 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:30.628995 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:30.628977 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:30.632412 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.632379 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nzmrx"] Sep 07 08:59:30.632602 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.632524 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:30.632668 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:30.632648 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:30.633168 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.633146 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kl8g9"] Sep 07 08:59:30.633246 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:30.633215 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:30.633289 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:30.633273 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:31.947136 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:31.946914 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:31.947645 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:31.946968 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:31.947645 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:31.947241 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:31.947645 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:31.947271 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:32.117200 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:32.117135 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdb82087-8753-4795-80ff-c9eae563fcc3" containerID="f9425dc35d354b0dcbe5e18d32454b85283f0ce0ba7cdce4f1697faf89c83c44" exitCode=0 Sep 07 08:59:32.117200 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:32.117189 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerDied","Data":"f9425dc35d354b0dcbe5e18d32454b85283f0ce0ba7cdce4f1697faf89c83c44"} Sep 07 08:59:32.945125 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:32.945099 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:32.945270 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:32.945207 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:33.945338 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:33.945286 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:33.945744 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:33.945418 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:33.945744 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:33.945474 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:33.945744 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:33.945603 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:34.123880 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:34.123851 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdb82087-8753-4795-80ff-c9eae563fcc3" containerID="0e37ff9c085179336e8f75d843f54bb045faa94b101b169d85dd8f1eb1590514" exitCode=0 Sep 07 08:59:34.124046 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:34.123886 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerDied","Data":"0e37ff9c085179336e8f75d843f54bb045faa94b101b169d85dd8f1eb1590514"} Sep 07 08:59:34.945786 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:34.945755 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:34.946261 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:34.945875 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:35.602208 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:35.602176 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:35.602531 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.602384 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 07 08:59:35.602531 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.602409 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 07 08:59:35.602531 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.602420 2558 projected.go:196] Error preparing data for projected volume kube-api-access-5g66j for pod openshift-network-diagnostics/network-check-target-kl8g9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:35.602531 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.602485 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j podName:854c6727-7f69-4d5f-bf0c-ade9ed39908a nodeName:}" failed. No retries permitted until 2026-09-07 09:00:07.602464694 +0000 UTC m=+66.119255578 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5g66j" (UniqueName: "kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j") pod "network-check-target-kl8g9" (UID: "854c6727-7f69-4d5f-bf0c-ade9ed39908a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 07 08:59:35.703445 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:35.703409 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:35.703620 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:35.703456 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:35.703620 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.703555 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:35.703620 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.703571 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:35.703746 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.703624 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret podName:41bba008-7502-4a97-8759-27e21db291c1 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:07.703605724 +0000 UTC m=+66.220396621 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret") pod "global-pull-secret-syncer-mhtrz" (UID: "41bba008-7502-4a97-8759-27e21db291c1") : object "kube-system"/"original-pull-secret" not registered Sep 07 08:59:35.703746 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.703649 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs podName:3ce552e2-a957-4a53-b836-e8ffec7bfb37 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:07.703638327 +0000 UTC m=+66.220429221 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs") pod "network-metrics-daemon-nzmrx" (UID: "3ce552e2-a957-4a53-b836-e8ffec7bfb37") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 07 08:59:35.945560 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:35.945472 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:35.945560 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:35.945498 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:35.945770 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.945607 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:35.945770 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:35.945708 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:36.945072 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:36.945041 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:36.945248 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:36.945167 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:37.945584 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:37.945547 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:37.946018 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:37.945660 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:37.946018 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:37.945716 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:37.946018 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:37.945841 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:38.135846 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:38.135800 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerStarted","Data":"5be55b5ad97d0fef88664612d77e7fa2e42af151cc4b761f6a59fa122c2adcaf"} Sep 07 08:59:38.945155 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:38.944986 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:38.945321 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:38.945227 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:39.141414 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:39.141379 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdb82087-8753-4795-80ff-c9eae563fcc3" containerID="5be55b5ad97d0fef88664612d77e7fa2e42af151cc4b761f6a59fa122c2adcaf" exitCode=0 Sep 07 08:59:39.141742 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:39.141442 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerDied","Data":"5be55b5ad97d0fef88664612d77e7fa2e42af151cc4b761f6a59fa122c2adcaf"} Sep 07 08:59:39.945603 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:39.945569 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:39.945781 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:39.945571 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:39.945781 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:39.945684 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:39.945781 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:39.945747 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:40.147287 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:40.147209 2558 generic.go:356] "Generic (PLEG): container finished" podID="bdb82087-8753-4795-80ff-c9eae563fcc3" containerID="1e40f4138a532219f3165d421cbcbe2d2debf5210da9e147be5bc8641406ad1a" exitCode=0 Sep 07 08:59:40.147287 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:40.147270 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerDied","Data":"1e40f4138a532219f3165d421cbcbe2d2debf5210da9e147be5bc8641406ad1a"} Sep 07 08:59:40.945627 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:40.945600 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:40.945779 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:40.945702 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:41.152771 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:41.152742 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" event={"ID":"bdb82087-8753-4795-80ff-c9eae563fcc3","Type":"ContainerStarted","Data":"06f5ff0b35d951a28ab2ed166074cfb532ee9b85edf71ec8761beb719a577612"} Sep 07 08:59:41.173361 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:41.173316 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mjtz8" podStartSLOduration=11.36143831 podStartE2EDuration="40.173285927s" totalImagesPullingTime="28.811847617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:41.172974256 +0000 UTC m=+39.689765162" watchObservedRunningTime="2026-09-07 08:59:41.173285927 +0000 UTC m=+39.690076853" Sep 07 08:59:41.945995 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:41.945970 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:41.946137 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:41.946058 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:41.946195 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:41.946161 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:41.946321 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:41.946285 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:42.945038 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:42.945010 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:42.945414 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:42.945109 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:43.945378 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:43.945350 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:43.945733 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:43.945450 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:43.945733 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:43.945477 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:43.945733 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:43.945543 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:44.945872 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:44.945846 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:44.946220 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:44.945943 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:45.945620 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:45.945595 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:45.945766 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:45.945704 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:45.945814 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:45.945777 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:45.945879 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:45.945862 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:46.945311 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:46.945259 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:46.945690 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:46.945397 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:47.945451 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:47.945410 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:47.945915 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:47.945411 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:47.945915 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:47.945520 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:47.945915 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:47.945589 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:48.945250 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:48.945213 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:48.945437 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:48.945349 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:49.945208 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:49.945175 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:49.945592 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:49.945281 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:49.945592 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:49.945353 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:49.945592 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:49.945432 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:50.945826 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:50.945797 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:50.946178 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:50.945886 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl8g9" podUID="854c6727-7f69-4d5f-bf0c-ade9ed39908a" Sep 07 08:59:51.947431 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:51.947394 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:51.947998 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:51.947537 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mhtrz" podUID="41bba008-7502-4a97-8759-27e21db291c1" Sep 07 08:59:51.947998 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:51.947830 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:51.947998 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:51.947965 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzmrx" podUID="3ce552e2-a957-4a53-b836-e8ffec7bfb37" Sep 07 08:59:52.843391 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.843360 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-66.ec2.internal" event="NodeReady" Sep 07 08:59:52.843553 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.843462 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 07 08:59:52.893113 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.893067 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n84s7"] Sep 07 08:59:52.904167 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.904142 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j45pg"] Sep 07 08:59:52.904280 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.904259 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:52.906807 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.906783 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hmltn\"" Sep 07 08:59:52.906968 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.906948 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 07 08:59:52.907035 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.906974 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 07 08:59:52.916909 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.916849 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n84s7"] Sep 07 08:59:52.917002 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.916919 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:52.917002 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.916939 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j45pg"] Sep 07 08:59:52.917002 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.916969 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-582vt"] Sep 07 08:59:52.919547 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.919522 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 07 08:59:52.919547 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.919540 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vhtvf\"" Sep 07 08:59:52.919702 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.919527 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 07 08:59:52.919702 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.919616 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 07 08:59:52.919864 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.919847 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 07 08:59:52.928090 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.928073 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-582vt"] Sep 07 08:59:52.928170 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.928151 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:52.930503 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.930486 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 07 08:59:52.930598 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.930584 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 07 08:59:52.930694 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.930681 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fc2n8\"" Sep 07 08:59:52.930728 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.930706 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 07 08:59:52.945399 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.945379 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 08:59:52.947816 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.947797 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 08:59:52.948137 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.947833 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlzfg\"" Sep 07 08:59:52.948137 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:52.947842 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 08:59:53.022179 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022153 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab84316f-177d-4c97-b2fd-c630e9a62af2-data-volume\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.022289 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022181 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl8ps\" (UniqueName: \"kubernetes.io/projected/ab84316f-177d-4c97-b2fd-c630e9a62af2-kube-api-access-bl8ps\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.022289 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022202 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n957n\" (UniqueName: \"kubernetes.io/projected/783315ec-087a-439f-bad8-138553f6413a-kube-api-access-n957n\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.022289 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022220 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/783315ec-087a-439f-bad8-138553f6413a-metrics-tls\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.022289 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022260 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab84316f-177d-4c97-b2fd-c630e9a62af2-crio-socket\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.022471 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022311 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/783315ec-087a-439f-bad8-138553f6413a-config-volume\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.022471 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022333 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab84316f-177d-4c97-b2fd-c630e9a62af2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.022471 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022369 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab84316f-177d-4c97-b2fd-c630e9a62af2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.022471 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.022410 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783315ec-087a-439f-bad8-138553f6413a-tmp-dir\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.123596 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123529 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783315ec-087a-439f-bad8-138553f6413a-tmp-dir\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.123596 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwj98\" (UniqueName: \"kubernetes.io/projected/cde1e62f-e104-437f-82ad-fd68be33a484-kube-api-access-jwj98\") pod \"ingress-canary-582vt\" (UID: \"cde1e62f-e104-437f-82ad-fd68be33a484\") " pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:53.123596 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123585 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cde1e62f-e104-437f-82ad-fd68be33a484-cert\") pod \"ingress-canary-582vt\" (UID: \"cde1e62f-e104-437f-82ad-fd68be33a484\") " pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:53.123837 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123604 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab84316f-177d-4c97-b2fd-c630e9a62af2-data-volume\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.123837 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123660 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl8ps\" (UniqueName: \"kubernetes.io/projected/ab84316f-177d-4c97-b2fd-c630e9a62af2-kube-api-access-bl8ps\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.123837 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n957n\" (UniqueName: \"kubernetes.io/projected/783315ec-087a-439f-bad8-138553f6413a-kube-api-access-n957n\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.123837 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123769 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/783315ec-087a-439f-bad8-138553f6413a-metrics-tls\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.123837 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123790 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab84316f-177d-4c97-b2fd-c630e9a62af2-crio-socket\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.123837 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123807 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/783315ec-087a-439f-bad8-138553f6413a-config-volume\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.124116 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123870 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab84316f-177d-4c97-b2fd-c630e9a62af2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.124116 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123902 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab84316f-177d-4c97-b2fd-c630e9a62af2-data-volume\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.124116 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123914 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab84316f-177d-4c97-b2fd-c630e9a62af2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.124116 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123935 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783315ec-087a-439f-bad8-138553f6413a-tmp-dir\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.124116 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.123952 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ab84316f-177d-4c97-b2fd-c630e9a62af2-crio-socket\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.124386 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.124368 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ab84316f-177d-4c97-b2fd-c630e9a62af2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.124386 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.124377 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/783315ec-087a-439f-bad8-138553f6413a-config-volume\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.127619 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.127595 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ab84316f-177d-4c97-b2fd-c630e9a62af2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.127735 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.127597 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/783315ec-087a-439f-bad8-138553f6413a-metrics-tls\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.131085 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.131059 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n957n\" (UniqueName: \"kubernetes.io/projected/783315ec-087a-439f-bad8-138553f6413a-kube-api-access-n957n\") pod \"dns-default-n84s7\" (UID: \"783315ec-087a-439f-bad8-138553f6413a\") " pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.131191 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.131120 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl8ps\" (UniqueName: \"kubernetes.io/projected/ab84316f-177d-4c97-b2fd-c630e9a62af2-kube-api-access-bl8ps\") pod \"insights-runtime-extractor-j45pg\" (UID: \"ab84316f-177d-4c97-b2fd-c630e9a62af2\") " pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.214282 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.214250 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:53.225039 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.225018 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j45pg" Sep 07 08:59:53.225148 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.225128 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwj98\" (UniqueName: \"kubernetes.io/projected/cde1e62f-e104-437f-82ad-fd68be33a484-kube-api-access-jwj98\") pod \"ingress-canary-582vt\" (UID: \"cde1e62f-e104-437f-82ad-fd68be33a484\") " pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:53.225186 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.225166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cde1e62f-e104-437f-82ad-fd68be33a484-cert\") pod \"ingress-canary-582vt\" (UID: \"cde1e62f-e104-437f-82ad-fd68be33a484\") " pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:53.227367 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.227344 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cde1e62f-e104-437f-82ad-fd68be33a484-cert\") pod \"ingress-canary-582vt\" (UID: \"cde1e62f-e104-437f-82ad-fd68be33a484\") " pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:53.232896 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.232871 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwj98\" (UniqueName: \"kubernetes.io/projected/cde1e62f-e104-437f-82ad-fd68be33a484-kube-api-access-jwj98\") pod \"ingress-canary-582vt\" (UID: \"cde1e62f-e104-437f-82ad-fd68be33a484\") " pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:53.236100 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.236080 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-582vt" Sep 07 08:59:53.517600 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.517527 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n84s7"] Sep 07 08:59:53.520699 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:53.520675 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783315ec_087a_439f_bad8_138553f6413a.slice/crio-796dd787f8315e698b5e2fa8488d1dba425c334ec96dbd4933411f5ec5cb66a3 WatchSource:0}: Error finding container 796dd787f8315e698b5e2fa8488d1dba425c334ec96dbd4933411f5ec5cb66a3: Status 404 returned error can't find the container with id 796dd787f8315e698b5e2fa8488d1dba425c334ec96dbd4933411f5ec5cb66a3 Sep 07 08:59:53.550623 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.550597 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-582vt"] Sep 07 08:59:53.553447 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:53.553417 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde1e62f_e104_437f_82ad_fd68be33a484.slice/crio-ac4e48ee8c9f37835511058ee92cf9832d64cacdcfc95bd28806bfd124828aa0 WatchSource:0}: Error finding container ac4e48ee8c9f37835511058ee92cf9832d64cacdcfc95bd28806bfd124828aa0: Status 404 returned error can't find the container with id ac4e48ee8c9f37835511058ee92cf9832d64cacdcfc95bd28806bfd124828aa0 Sep 07 08:59:53.553779 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.553758 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j45pg"] Sep 07 08:59:53.556598 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:53.556578 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab84316f_177d_4c97_b2fd_c630e9a62af2.slice/crio-f2b8fbfa36bcf83a2a347da87c0c3021748faac2ccd2fe8e63e457d5d221b3cd WatchSource:0}: Error finding container f2b8fbfa36bcf83a2a347da87c0c3021748faac2ccd2fe8e63e457d5d221b3cd: Status 404 returned error can't find the container with id f2b8fbfa36bcf83a2a347da87c0c3021748faac2ccd2fe8e63e457d5d221b3cd Sep 07 08:59:53.646292 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.646263 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w"] Sep 07 08:59:53.655153 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.655134 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:53.657745 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.657722 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 07 08:59:53.657956 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.657939 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-75tjk\"" Sep 07 08:59:53.658615 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.658592 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w"] Sep 07 08:59:53.728637 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.728600 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/021ea2e1-192d-45da-9f14-715952579103-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vvw2w\" (UID: \"021ea2e1-192d-45da-9f14-715952579103\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:53.828955 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.828918 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/021ea2e1-192d-45da-9f14-715952579103-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vvw2w\" (UID: \"021ea2e1-192d-45da-9f14-715952579103\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:53.829091 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:53.829066 2558 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 07 08:59:53.829144 ip-10-0-131-66 kubenswrapper[2558]: E0907 08:59:53.829136 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/021ea2e1-192d-45da-9f14-715952579103-tls-certificates podName:021ea2e1-192d-45da-9f14-715952579103 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:54.329118244 +0000 UTC m=+52.845909127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/021ea2e1-192d-45da-9f14-715952579103-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-vvw2w" (UID: "021ea2e1-192d-45da-9f14-715952579103") : secret "prometheus-operator-admission-webhook-tls" not found Sep 07 08:59:53.945338 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.945285 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 08:59:53.945503 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.945352 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 08:59:53.948021 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.947987 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 08:59:53.948397 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.947991 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 08:59:53.948397 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:53.948078 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-txgtt\"" Sep 07 08:59:54.182969 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.182882 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n84s7" event={"ID":"783315ec-087a-439f-bad8-138553f6413a","Type":"ContainerStarted","Data":"796dd787f8315e698b5e2fa8488d1dba425c334ec96dbd4933411f5ec5cb66a3"} Sep 07 08:59:54.184539 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.184509 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-582vt" event={"ID":"cde1e62f-e104-437f-82ad-fd68be33a484","Type":"ContainerStarted","Data":"ac4e48ee8c9f37835511058ee92cf9832d64cacdcfc95bd28806bfd124828aa0"} Sep 07 08:59:54.186645 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.186617 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j45pg" event={"ID":"ab84316f-177d-4c97-b2fd-c630e9a62af2","Type":"ContainerStarted","Data":"fdd26a087e0f87f89d3df73adb5b31435f3c2f430a3c15156e74168a5b06c04a"} Sep 07 08:59:54.186739 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.186674 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j45pg" event={"ID":"ab84316f-177d-4c97-b2fd-c630e9a62af2","Type":"ContainerStarted","Data":"f2b8fbfa36bcf83a2a347da87c0c3021748faac2ccd2fe8e63e457d5d221b3cd"} Sep 07 08:59:54.332998 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.332961 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/021ea2e1-192d-45da-9f14-715952579103-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vvw2w\" (UID: \"021ea2e1-192d-45da-9f14-715952579103\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:54.336218 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.336185 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/021ea2e1-192d-45da-9f14-715952579103-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vvw2w\" (UID: \"021ea2e1-192d-45da-9f14-715952579103\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:54.564958 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.564913 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:54.875957 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:54.875885 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w"] Sep 07 08:59:55.191766 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:55.191676 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j45pg" event={"ID":"ab84316f-177d-4c97-b2fd-c630e9a62af2","Type":"ContainerStarted","Data":"5a3110187584b94d7e4486c3cd9f23e4f923f5a6543314e4fe32c6965ce92198"} Sep 07 08:59:55.297617 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:55.297586 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021ea2e1_192d_45da_9f14_715952579103.slice/crio-33641004d88597c27746ad5ef23bf02e27826dcbfb843b233f845e18947dba21 WatchSource:0}: Error finding container 33641004d88597c27746ad5ef23bf02e27826dcbfb843b233f845e18947dba21: Status 404 returned error can't find the container with id 33641004d88597c27746ad5ef23bf02e27826dcbfb843b233f845e18947dba21 Sep 07 08:59:56.199698 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:56.199656 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-582vt" event={"ID":"cde1e62f-e104-437f-82ad-fd68be33a484","Type":"ContainerStarted","Data":"e43c307c5491f48e0ccb75103d95689267d91d5b32060550a2f0a062cc83dcfe"} Sep 07 08:59:56.202558 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:56.202526 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n84s7" event={"ID":"783315ec-087a-439f-bad8-138553f6413a","Type":"ContainerStarted","Data":"26a61323b09de5f02d4e4b321c548f71496209f387a7f4e8d573a07fd286dd50"} Sep 07 08:59:56.207174 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:56.206391 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" event={"ID":"021ea2e1-192d-45da-9f14-715952579103","Type":"ContainerStarted","Data":"33641004d88597c27746ad5ef23bf02e27826dcbfb843b233f845e18947dba21"} Sep 07 08:59:56.215005 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:56.214955 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-582vt" podStartSLOduration=1.723104061 podStartE2EDuration="4.214937174s" totalImagesPullingTime="2.491833113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:56.213848033 +0000 UTC m=+54.730638944" watchObservedRunningTime="2026-09-07 08:59:56.214937174 +0000 UTC m=+54.731728079" Sep 07 08:59:57.211847 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.211818 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j45pg" event={"ID":"ab84316f-177d-4c97-b2fd-c630e9a62af2","Type":"ContainerStarted","Data":"44fd4ed5f264615b8066741a725046d2ccfa488f58b3e75369ac2159fc568c74"} Sep 07 08:59:57.213678 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.213652 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n84s7" event={"ID":"783315ec-087a-439f-bad8-138553f6413a","Type":"ContainerStarted","Data":"2e5de8d65e7d15e4a8d21ad5f76edf065332b5df6887066a9200f91ffcd0da5d"} Sep 07 08:59:57.213796 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.213713 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n84s7" Sep 07 08:59:57.215237 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.215213 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" event={"ID":"021ea2e1-192d-45da-9f14-715952579103","Type":"ContainerStarted","Data":"a3868f8d7b6497bc18559a35354b60137987a84174f2cbca022f2249cc826bd0"} Sep 07 08:59:57.215409 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.215392 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:57.216668 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.216646 2558 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-78f599db79-vvw2w container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.133.0.9:8443/healthz\": dial tcp 10.133.0.9:8443: connect: connection refused" start-of-body= Sep 07 08:59:57.216767 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.216689 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" podUID="021ea2e1-192d-45da-9f14-715952579103" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.133.0.9:8443/healthz\": dial tcp 10.133.0.9:8443: connect: connection refused" Sep 07 08:59:57.230240 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.230193 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j45pg" podStartSLOduration=1.812656791 podStartE2EDuration="5.230179857s" totalImagesPullingTime="3.417523066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:57.229408395 +0000 UTC m=+55.746199298" watchObservedRunningTime="2026-09-07 08:59:57.230179857 +0000 UTC m=+55.746970759" Sep 07 08:59:57.245575 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.245504 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n84s7" podStartSLOduration=2.725225465 podStartE2EDuration="5.245492602s" totalImagesPullingTime="2.520267137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:57.244538693 +0000 UTC m=+55.761329597" watchObservedRunningTime="2026-09-07 08:59:57.245492602 +0000 UTC m=+55.762283505" Sep 07 08:59:57.259163 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:57.259131 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" podStartSLOduration=2.416513544 podStartE2EDuration="4.259120415s" totalImagesPullingTime="1.842606871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:57.258044305 +0000 UTC m=+55.774835208" watchObservedRunningTime="2026-09-07 08:59:57.259120415 +0000 UTC m=+55.775911318" Sep 07 08:59:58.222363 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.222331 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vvw2w" Sep 07 08:59:58.708310 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.708263 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-4c4kb"] Sep 07 08:59:58.711401 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.711385 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.713804 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.713784 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 07 08:59:58.714742 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.714724 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 07 08:59:58.714830 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.714812 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 07 08:59:58.714884 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.714833 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 07 08:59:58.714884 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.714848 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 07 08:59:58.714884 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.714832 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tjvpn\"" Sep 07 08:59:58.722287 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.722250 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-4c4kb"] Sep 07 08:59:58.768998 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.768975 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b029d92e-5d09-4a55-9199-7cb44eb1f552-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.769110 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.769025 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b029d92e-5d09-4a55-9199-7cb44eb1f552-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.769110 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.769053 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b029d92e-5d09-4a55-9199-7cb44eb1f552-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.769193 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.769151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh4vs\" (UniqueName: \"kubernetes.io/projected/b029d92e-5d09-4a55-9199-7cb44eb1f552-kube-api-access-hh4vs\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.869542 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.869516 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b029d92e-5d09-4a55-9199-7cb44eb1f552-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.869641 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.869555 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b029d92e-5d09-4a55-9199-7cb44eb1f552-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.869641 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.869574 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b029d92e-5d09-4a55-9199-7cb44eb1f552-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.869748 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.869727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh4vs\" (UniqueName: \"kubernetes.io/projected/b029d92e-5d09-4a55-9199-7cb44eb1f552-kube-api-access-hh4vs\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.870259 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.870240 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b029d92e-5d09-4a55-9199-7cb44eb1f552-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.872009 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.871984 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b029d92e-5d09-4a55-9199-7cb44eb1f552-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.872088 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.872073 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b029d92e-5d09-4a55-9199-7cb44eb1f552-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:58.877541 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:58.877522 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh4vs\" (UniqueName: \"kubernetes.io/projected/b029d92e-5d09-4a55-9199-7cb44eb1f552-kube-api-access-hh4vs\") pod \"prometheus-operator-6f4946db56-4c4kb\" (UID: \"b029d92e-5d09-4a55-9199-7cb44eb1f552\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:59.020093 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:59.020064 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" Sep 07 08:59:59.286416 ip-10-0-131-66 kubenswrapper[2558]: I0907 08:59:59.286321 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-4c4kb"] Sep 07 08:59:59.290416 ip-10-0-131-66 kubenswrapper[2558]: W0907 08:59:59.290389 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb029d92e_5d09_4a55_9199_7cb44eb1f552.slice/crio-8f6e24703295935c71e2473236472203e34468b72021684c835f28b354e9fac8 WatchSource:0}: Error finding container 8f6e24703295935c71e2473236472203e34468b72021684c835f28b354e9fac8: Status 404 returned error can't find the container with id 8f6e24703295935c71e2473236472203e34468b72021684c835f28b354e9fac8 Sep 07 09:00:00.227942 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:00.227903 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" event={"ID":"b029d92e-5d09-4a55-9199-7cb44eb1f552","Type":"ContainerStarted","Data":"8f6e24703295935c71e2473236472203e34468b72021684c835f28b354e9fac8"} Sep 07 09:00:01.133357 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:01.133331 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5gdns" Sep 07 09:00:01.236161 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:01.236125 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" event={"ID":"b029d92e-5d09-4a55-9199-7cb44eb1f552","Type":"ContainerStarted","Data":"1fc34deff6532082cf54f5c5e74787b10e3e0037ee95a77de8407a7759be75f6"} Sep 07 09:00:01.236354 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:01.236202 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" event={"ID":"b029d92e-5d09-4a55-9199-7cb44eb1f552","Type":"ContainerStarted","Data":"6e7861e86da69f5873525bf4ebcb0d2ddaaf7ba0581190a48fb74daf1780c7fc"} Sep 07 09:00:01.253774 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:01.253721 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-4c4kb" podStartSLOduration=1.7788761690000001 podStartE2EDuration="3.253703524s" totalImagesPullingTime="1.474827355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:01.252985571 +0000 UTC m=+59.769776513" watchObservedRunningTime="2026-09-07 09:00:01.253703524 +0000 UTC m=+59.770494434" Sep 07 09:00:03.067353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.067319 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr"] Sep 07 09:00:03.070311 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.070280 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.072941 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.072920 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-btl4t\"" Sep 07 09:00:03.073033 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.072921 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 07 09:00:03.073086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.072926 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 07 09:00:03.077709 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.077690 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hv7zr"] Sep 07 09:00:03.081254 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.081235 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.084126 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.084105 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr"] Sep 07 09:00:03.084772 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.084745 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 07 09:00:03.084772 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.084767 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 07 09:00:03.084906 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.084772 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jcq2k\"" Sep 07 09:00:03.085084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.085065 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 07 09:00:03.087312 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.087277 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw"] Sep 07 09:00:03.089389 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.089365 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.092127 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.092107 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 07 09:00:03.092205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.092194 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 07 09:00:03.092557 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.092539 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 07 09:00:03.092604 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.092567 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4wjb9\"" Sep 07 09:00:03.096953 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.096935 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-accelerators-collector-config\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.096961 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-tls\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.096979 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-textfile\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.096995 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433411a-91b1-4311-bb6a-51c5aa61d9e2-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.097137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-root\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097075 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/755ef248-b213-45b9-aa97-003e40c24356-metrics-client-ca\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097106 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-sys\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097235 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097137 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxwx4\" (UniqueName: \"kubernetes.io/projected/2433411a-91b1-4311-bb6a-51c5aa61d9e2-kube-api-access-pxwx4\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.097235 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097180 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-wtmp\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097235 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097203 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.097347 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8smbp\" (UniqueName: \"kubernetes.io/projected/755ef248-b213-45b9-aa97-003e40c24356-kube-api-access-8smbp\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.097347 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097260 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.097347 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.097341 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.105901 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.105881 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw"] Sep 07 09:00:03.197613 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197577 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-textfile\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.197613 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197611 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433411a-91b1-4311-bb6a-51c5aa61d9e2-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197646 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-root\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197663 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/755ef248-b213-45b9-aa97-003e40c24356-metrics-client-ca\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197686 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-sys\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197709 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxwx4\" (UniqueName: \"kubernetes.io/projected/2433411a-91b1-4311-bb6a-51c5aa61d9e2-kube-api-access-pxwx4\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197725 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-root\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197781 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-sys\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197821 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-wtmp\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197845 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.197880 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197873 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8smbp\" (UniqueName: \"kubernetes.io/projected/755ef248-b213-45b9-aa97-003e40c24356-kube-api-access-8smbp\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197896 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197931 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.197979 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198004 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-wtmp\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198035 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-textfile\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198034 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8mgk\" (UniqueName: \"kubernetes.io/projected/c0159467-5645-46ef-bc19-96fd5967324d-kube-api-access-c8mgk\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:00:03.198076 2558 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:00:03.198127 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-tls podName:2433411a-91b1-4311-bb6a-51c5aa61d9e2 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:03.698110975 +0000 UTC m=+62.214901856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-4wswr" (UID: "2433411a-91b1-4311-bb6a-51c5aa61d9e2") : secret "openshift-state-metrics-tls" not found Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198156 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198200 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c0159467-5645-46ef-bc19-96fd5967324d-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198242 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-accelerators-collector-config\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198274 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-tls\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.198342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198321 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0159467-5645-46ef-bc19-96fd5967324d-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.198830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198384 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/755ef248-b213-45b9-aa97-003e40c24356-metrics-client-ca\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.198830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198439 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2433411a-91b1-4311-bb6a-51c5aa61d9e2-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.198830 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:00:03.198442 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 07 09:00:03.198830 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:00:03.198541 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-tls podName:755ef248-b213-45b9-aa97-003e40c24356 nodeName:}" failed. No retries permitted until 2026-09-07 09:00:03.698524211 +0000 UTC m=+62.215315107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-tls") pod "node-exporter-hv7zr" (UID: "755ef248-b213-45b9-aa97-003e40c24356") : secret "node-exporter-tls" not found Sep 07 09:00:03.198830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.198686 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-accelerators-collector-config\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.200434 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.200418 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.200543 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.200523 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.205787 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.205766 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8smbp\" (UniqueName: \"kubernetes.io/projected/755ef248-b213-45b9-aa97-003e40c24356-kube-api-access-8smbp\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.205911 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.205893 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxwx4\" (UniqueName: \"kubernetes.io/projected/2433411a-91b1-4311-bb6a-51c5aa61d9e2-kube-api-access-pxwx4\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.298854 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.298816 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.299011 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.298865 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.299011 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.298913 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8mgk\" (UniqueName: \"kubernetes.io/projected/c0159467-5645-46ef-bc19-96fd5967324d-kube-api-access-c8mgk\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.299011 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.298957 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.299011 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:00:03.298972 2558 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 07 09:00:03.299011 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.298999 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c0159467-5645-46ef-bc19-96fd5967324d-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.299222 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:00:03.299044 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-tls podName:c0159467-5645-46ef-bc19-96fd5967324d nodeName:}" failed. No retries permitted until 2026-09-07 09:00:03.7990252 +0000 UTC m=+62.315816101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-7ftnw" (UID: "c0159467-5645-46ef-bc19-96fd5967324d") : secret "kube-state-metrics-tls" not found Sep 07 09:00:03.299222 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.299082 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0159467-5645-46ef-bc19-96fd5967324d-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.299572 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.299548 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c0159467-5645-46ef-bc19-96fd5967324d-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.299967 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.299946 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0159467-5645-46ef-bc19-96fd5967324d-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.300043 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.300018 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.301413 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.301395 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.310183 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.310158 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8mgk\" (UniqueName: \"kubernetes.io/projected/c0159467-5645-46ef-bc19-96fd5967324d-kube-api-access-c8mgk\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.646542 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.646506 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-7dqmf"] Sep 07 09:00:03.649692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.649677 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-7dqmf" Sep 07 09:00:03.652238 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.652216 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5sgwq\"" Sep 07 09:00:03.652373 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.652238 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 07 09:00:03.652490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.652477 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 07 09:00:03.664809 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.664784 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-7dqmf"] Sep 07 09:00:03.702566 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.702537 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.702699 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.702629 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-tls\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.702699 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.702661 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmbxl\" (UniqueName: \"kubernetes.io/projected/fdd66e31-1fc9-41c2-8c70-d5dd7c29d836-kube-api-access-pmbxl\") pod \"downloads-85d98c998f-7dqmf\" (UID: \"fdd66e31-1fc9-41c2-8c70-d5dd7c29d836\") " pod="openshift-console/downloads-85d98c998f-7dqmf" Sep 07 09:00:03.704886 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.704857 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/755ef248-b213-45b9-aa97-003e40c24356-node-exporter-tls\") pod \"node-exporter-hv7zr\" (UID: \"755ef248-b213-45b9-aa97-003e40c24356\") " pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.704975 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.704948 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2433411a-91b1-4311-bb6a-51c5aa61d9e2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-4wswr\" (UID: \"2433411a-91b1-4311-bb6a-51c5aa61d9e2\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.803473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.803432 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmbxl\" (UniqueName: \"kubernetes.io/projected/fdd66e31-1fc9-41c2-8c70-d5dd7c29d836-kube-api-access-pmbxl\") pod \"downloads-85d98c998f-7dqmf\" (UID: \"fdd66e31-1fc9-41c2-8c70-d5dd7c29d836\") " pod="openshift-console/downloads-85d98c998f-7dqmf" Sep 07 09:00:03.803632 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.803493 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.805718 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.805692 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c0159467-5645-46ef-bc19-96fd5967324d-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7ftnw\" (UID: \"c0159467-5645-46ef-bc19-96fd5967324d\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:03.811195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.811177 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmbxl\" (UniqueName: \"kubernetes.io/projected/fdd66e31-1fc9-41c2-8c70-d5dd7c29d836-kube-api-access-pmbxl\") pod \"downloads-85d98c998f-7dqmf\" (UID: \"fdd66e31-1fc9-41c2-8c70-d5dd7c29d836\") " pod="openshift-console/downloads-85d98c998f-7dqmf" Sep 07 09:00:03.958456 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.958380 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-7dqmf" Sep 07 09:00:03.979812 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.979784 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" Sep 07 09:00:03.993716 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.993679 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hv7zr" Sep 07 09:00:03.999150 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:03.998728 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" Sep 07 09:00:04.013123 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:04.012312 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755ef248_b213_45b9_aa97_003e40c24356.slice/crio-bdcf0e5ad68d4e24273c80122c502f113804afcfd5a760ad84d887e0eb93f524 WatchSource:0}: Error finding container bdcf0e5ad68d4e24273c80122c502f113804afcfd5a760ad84d887e0eb93f524: Status 404 returned error can't find the container with id bdcf0e5ad68d4e24273c80122c502f113804afcfd5a760ad84d887e0eb93f524 Sep 07 09:00:04.233604 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:04.233534 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-7dqmf"] Sep 07 09:00:04.236504 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:04.236480 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd66e31_1fc9_41c2_8c70_d5dd7c29d836.slice/crio-28ff07d1075fb3643c5e57280041eb7c592410508ddc239fb5d3d0cd1c7be7a5 WatchSource:0}: Error finding container 28ff07d1075fb3643c5e57280041eb7c592410508ddc239fb5d3d0cd1c7be7a5: Status 404 returned error can't find the container with id 28ff07d1075fb3643c5e57280041eb7c592410508ddc239fb5d3d0cd1c7be7a5 Sep 07 09:00:04.246105 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:04.246080 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hv7zr" event={"ID":"755ef248-b213-45b9-aa97-003e40c24356","Type":"ContainerStarted","Data":"bdcf0e5ad68d4e24273c80122c502f113804afcfd5a760ad84d887e0eb93f524"} Sep 07 09:00:04.247138 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:04.247117 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-7dqmf" event={"ID":"fdd66e31-1fc9-41c2-8c70-d5dd7c29d836","Type":"ContainerStarted","Data":"28ff07d1075fb3643c5e57280041eb7c592410508ddc239fb5d3d0cd1c7be7a5"} Sep 07 09:00:04.268390 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:04.268359 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr"] Sep 07 09:00:04.270868 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:04.270844 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2433411a_91b1_4311_bb6a_51c5aa61d9e2.slice/crio-f4a7d4e9783fb60e4706afdbd27554a0f9ef215584ed0114502a4b8e3ec54739 WatchSource:0}: Error finding container f4a7d4e9783fb60e4706afdbd27554a0f9ef215584ed0114502a4b8e3ec54739: Status 404 returned error can't find the container with id f4a7d4e9783fb60e4706afdbd27554a0f9ef215584ed0114502a4b8e3ec54739 Sep 07 09:00:04.299218 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:04.299188 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw"] Sep 07 09:00:04.303474 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:04.303440 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0159467_5645_46ef_bc19_96fd5967324d.slice/crio-bcfe02a82fe0dca9f858a56f7b1dc07d456f9161e999e24c24a16a20ce36e20e WatchSource:0}: Error finding container bcfe02a82fe0dca9f858a56f7b1dc07d456f9161e999e24c24a16a20ce36e20e: Status 404 returned error can't find the container with id bcfe02a82fe0dca9f858a56f7b1dc07d456f9161e999e24c24a16a20ce36e20e Sep 07 09:00:05.253942 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:05.253901 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" event={"ID":"c0159467-5645-46ef-bc19-96fd5967324d","Type":"ContainerStarted","Data":"bcfe02a82fe0dca9f858a56f7b1dc07d456f9161e999e24c24a16a20ce36e20e"} Sep 07 09:00:05.256480 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:05.256451 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" event={"ID":"2433411a-91b1-4311-bb6a-51c5aa61d9e2","Type":"ContainerStarted","Data":"0d35e48c7d41ae20ecb4faaaf4ae3fba95fa1bd8bd89bd2e57bf95aadef54b03"} Sep 07 09:00:05.256615 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:05.256516 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" event={"ID":"2433411a-91b1-4311-bb6a-51c5aa61d9e2","Type":"ContainerStarted","Data":"3a883152a4de858a27a8062e1385ab12760ed1df78e0ec5497053c32010ac421"} Sep 07 09:00:05.256615 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:05.256549 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" event={"ID":"2433411a-91b1-4311-bb6a-51c5aa61d9e2","Type":"ContainerStarted","Data":"f4a7d4e9783fb60e4706afdbd27554a0f9ef215584ed0114502a4b8e3ec54739"} Sep 07 09:00:06.263322 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:06.263264 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" event={"ID":"2433411a-91b1-4311-bb6a-51c5aa61d9e2","Type":"ContainerStarted","Data":"947d302a5206eda427ef6d10c1fe5cf56edf84763e9835dda04a3683bac957dc"} Sep 07 09:00:06.265644 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:06.265600 2558 generic.go:356] "Generic (PLEG): container finished" podID="755ef248-b213-45b9-aa97-003e40c24356" containerID="33c51e49cfea4b3fae9bfb5da6a5c3fdb0dfc73722c98c112b477f7fc89d8ca3" exitCode=0 Sep 07 09:00:06.265777 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:06.265647 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hv7zr" event={"ID":"755ef248-b213-45b9-aa97-003e40c24356","Type":"ContainerDied","Data":"33c51e49cfea4b3fae9bfb5da6a5c3fdb0dfc73722c98c112b477f7fc89d8ca3"} Sep 07 09:00:06.302361 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:06.302287 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-4wswr" podStartSLOduration=2.318251265 podStartE2EDuration="3.302264569s" totalImagesPullingTime="984.013304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:06.280269078 +0000 UTC m=+64.797059984" watchObservedRunningTime="2026-09-07 09:00:06.302264569 +0000 UTC m=+64.819055474" Sep 07 09:00:07.221726 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.221694 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n84s7" Sep 07 09:00:07.271975 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.271937 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hv7zr" event={"ID":"755ef248-b213-45b9-aa97-003e40c24356","Type":"ContainerStarted","Data":"40158352da04dd8aa174b7eb23a41b9e95ef994c4fe94d6d9153ee9718be8188"} Sep 07 09:00:07.272426 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.271990 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hv7zr" event={"ID":"755ef248-b213-45b9-aa97-003e40c24356","Type":"ContainerStarted","Data":"e71f36c0c2f2ce2ddd443fadc24334129a4777b0febfde8cb10f8a6ec03b81d0"} Sep 07 09:00:07.275819 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.275784 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" event={"ID":"c0159467-5645-46ef-bc19-96fd5967324d","Type":"ContainerStarted","Data":"0a49025b65d451b70d07274fb3ea8a9256e5887eb8e1d5bbde28b22d9d4410de"} Sep 07 09:00:07.276014 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.275882 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" event={"ID":"c0159467-5645-46ef-bc19-96fd5967324d","Type":"ContainerStarted","Data":"1ec1ae49fc6ff68aecf2f89ebee1f8db9c880101bd9800df5683942f8caa4007"} Sep 07 09:00:07.276014 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.275919 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" event={"ID":"c0159467-5645-46ef-bc19-96fd5967324d","Type":"ContainerStarted","Data":"b9d51439a0e137cb69ed13d0585383f0eefef3513f1d625f582aa3426c6a4ae8"} Sep 07 09:00:07.290516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.290460 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hv7zr" podStartSLOduration=2.89384665 podStartE2EDuration="4.290446177s" totalImagesPullingTime="1.319827376s" totalInitContainerRuntime="76.772151ms" isStatefulPod=true podCreationTimestamp="2026-09-07 09:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:07.289635872 +0000 UTC m=+65.806426776" watchObservedRunningTime="2026-09-07 09:00:07.290446177 +0000 UTC m=+65.807237080" Sep 07 09:00:07.307807 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.307738 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7ftnw" podStartSLOduration=2.155400148 podStartE2EDuration="4.307715195s" totalImagesPullingTime="2.152315047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:07.307206411 +0000 UTC m=+65.823997330" watchObservedRunningTime="2026-09-07 09:00:07.307715195 +0000 UTC m=+65.824506111" Sep 07 09:00:07.640697 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.640655 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 09:00:07.643338 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.643313 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 07 09:00:07.653752 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.653728 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 07 09:00:07.664534 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.664505 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g66j\" (UniqueName: \"kubernetes.io/projected/854c6727-7f69-4d5f-bf0c-ade9ed39908a-kube-api-access-5g66j\") pod \"network-check-target-kl8g9\" (UID: \"854c6727-7f69-4d5f-bf0c-ade9ed39908a\") " pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 09:00:07.741185 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.741148 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 09:00:07.741353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.741221 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 09:00:07.743535 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.743506 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 07 09:00:07.743672 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.743509 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 07 09:00:07.754764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.754741 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/41bba008-7502-4a97-8759-27e21db291c1-original-pull-secret\") pod \"global-pull-secret-syncer-mhtrz\" (UID: \"41bba008-7502-4a97-8759-27e21db291c1\") " pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 09:00:07.754883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.754801 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ce552e2-a957-4a53-b836-e8ffec7bfb37-metrics-certs\") pod \"network-metrics-daemon-nzmrx\" (UID: \"3ce552e2-a957-4a53-b836-e8ffec7bfb37\") " pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 09:00:07.758575 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.758551 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-txgtt\"" Sep 07 09:00:07.762386 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.762366 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mhtrz" Sep 07 09:00:07.767475 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.767459 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzmrx" Sep 07 09:00:07.955042 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.954955 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlzfg\"" Sep 07 09:00:07.963919 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:07.963896 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 09:00:08.076661 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:08.076632 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mhtrz"] Sep 07 09:00:08.082415 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:08.082359 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41bba008_7502_4a97_8759_27e21db291c1.slice/crio-ae8a8cabd0ee75ca1371f433d53d87f0086ea8c203496f06ab325b2a2b1ab4d0 WatchSource:0}: Error finding container ae8a8cabd0ee75ca1371f433d53d87f0086ea8c203496f06ab325b2a2b1ab4d0: Status 404 returned error can't find the container with id ae8a8cabd0ee75ca1371f433d53d87f0086ea8c203496f06ab325b2a2b1ab4d0 Sep 07 09:00:08.108382 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:08.108074 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nzmrx"] Sep 07 09:00:08.110803 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:08.110768 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce552e2_a957_4a53_b836_e8ffec7bfb37.slice/crio-7c80b58799ce19d5e2a5f803c0433d74bf0e47194b05c808a122863b27ab863b WatchSource:0}: Error finding container 7c80b58799ce19d5e2a5f803c0433d74bf0e47194b05c808a122863b27ab863b: Status 404 returned error can't find the container with id 7c80b58799ce19d5e2a5f803c0433d74bf0e47194b05c808a122863b27ab863b Sep 07 09:00:08.270566 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:08.270509 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kl8g9"] Sep 07 09:00:08.273275 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:08.273246 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod854c6727_7f69_4d5f_bf0c_ade9ed39908a.slice/crio-f7bd176896e6a0fec8932ad9635d1c4f2238d1a616c544a6842168aeda6edb82 WatchSource:0}: Error finding container f7bd176896e6a0fec8932ad9635d1c4f2238d1a616c544a6842168aeda6edb82: Status 404 returned error can't find the container with id f7bd176896e6a0fec8932ad9635d1c4f2238d1a616c544a6842168aeda6edb82 Sep 07 09:00:08.280260 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:08.280236 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nzmrx" event={"ID":"3ce552e2-a957-4a53-b836-e8ffec7bfb37","Type":"ContainerStarted","Data":"7c80b58799ce19d5e2a5f803c0433d74bf0e47194b05c808a122863b27ab863b"} Sep 07 09:00:08.281725 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:08.281692 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mhtrz" event={"ID":"41bba008-7502-4a97-8759-27e21db291c1","Type":"ContainerStarted","Data":"ae8a8cabd0ee75ca1371f433d53d87f0086ea8c203496f06ab325b2a2b1ab4d0"} Sep 07 09:00:08.282902 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:08.282873 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kl8g9" event={"ID":"854c6727-7f69-4d5f-bf0c-ade9ed39908a","Type":"ContainerStarted","Data":"f7bd176896e6a0fec8932ad9635d1c4f2238d1a616c544a6842168aeda6edb82"} Sep 07 09:00:10.298804 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:10.298758 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nzmrx" event={"ID":"3ce552e2-a957-4a53-b836-e8ffec7bfb37","Type":"ContainerStarted","Data":"56ba4f1c2fa2f8bbd6dbcc0c9f44476f70d4aa130d985e5da501819a2ca06a88"} Sep 07 09:00:10.299371 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:10.298836 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nzmrx" event={"ID":"3ce552e2-a957-4a53-b836-e8ffec7bfb37","Type":"ContainerStarted","Data":"987894b6f4350c3c068036cea6145e7a9c1cce6e5b4cba3b29f59dccfdf51da1"} Sep 07 09:00:10.317264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:10.317203 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nzmrx" podStartSLOduration=66.764720517 podStartE2EDuration="1m8.317180876s" totalImagesPullingTime="1.552460359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:10.314243269 +0000 UTC m=+68.831034173" watchObservedRunningTime="2026-09-07 09:00:10.317180876 +0000 UTC m=+68.833971779" Sep 07 09:00:14.316998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.316661 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kl8g9" event={"ID":"854c6727-7f69-4d5f-bf0c-ade9ed39908a","Type":"ContainerStarted","Data":"6f155f2fb619e5479e64219038921a25c515bbfb3b6a31e21dda2e953b5430c6"} Sep 07 09:00:14.316998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.316747 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 09:00:14.320206 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.320130 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mhtrz" event={"ID":"41bba008-7502-4a97-8759-27e21db291c1","Type":"ContainerStarted","Data":"a5481ec75af88c36910cb6c891979443a3975160609928b3851f276155beee62"} Sep 07 09:00:14.332158 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.331820 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kl8g9" podStartSLOduration=66.688267398 podStartE2EDuration="1m12.331799972s" totalImagesPullingTime="5.643532574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:14.331239018 +0000 UTC m=+72.848029921" watchObservedRunningTime="2026-09-07 09:00:14.331799972 +0000 UTC m=+72.848590876" Sep 07 09:00:14.346377 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.346317 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mhtrz" podStartSLOduration=66.510273292 podStartE2EDuration="1m12.346286293s" totalImagesPullingTime="5.836013001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:14.344787146 +0000 UTC m=+72.861578049" watchObservedRunningTime="2026-09-07 09:00:14.346286293 +0000 UTC m=+72.863077215" Sep 07 09:00:14.699874 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.699801 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-576c97495d-c67c9"] Sep 07 09:00:14.703001 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.702976 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.705398 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.705375 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 07 09:00:14.705591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.705568 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4vfbs\"" Sep 07 09:00:14.705706 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.705594 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 07 09:00:14.706461 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.706429 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 07 09:00:14.706564 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.706496 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 07 09:00:14.706624 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.706561 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 07 09:00:14.713172 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.713153 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576c97495d-c67c9"] Sep 07 09:00:14.804205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.804141 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-serving-cert\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.804403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.804226 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq854\" (UniqueName: \"kubernetes.io/projected/210f99b4-6ff3-40ee-acdd-de81a44519a4-kube-api-access-nq854\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.804403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.804259 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-config\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.804403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.804288 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-oauth-config\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.804403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.804384 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-service-ca\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.804672 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.804456 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-oauth-serving-cert\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.905788 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.905755 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq854\" (UniqueName: \"kubernetes.io/projected/210f99b4-6ff3-40ee-acdd-de81a44519a4-kube-api-access-nq854\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.905932 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.905798 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-config\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.905932 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.905829 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-oauth-config\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.905932 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.905865 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-service-ca\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.905932 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.905885 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-oauth-serving-cert\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.905932 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.905913 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-serving-cert\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.906911 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.906849 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-service-ca\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.906911 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.906851 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-config\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.906911 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.906851 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-oauth-serving-cert\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.908754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.908721 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-oauth-config\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.908754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.908740 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-serving-cert\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:14.922931 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:14.922881 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq854\" (UniqueName: \"kubernetes.io/projected/210f99b4-6ff3-40ee-acdd-de81a44519a4-kube-api-access-nq854\") pod \"console-576c97495d-c67c9\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:15.015177 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:15.015096 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:15.290323 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:15.290277 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576c97495d-c67c9"] Sep 07 09:00:22.256047 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:22.256012 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210f99b4_6ff3_40ee_acdd_de81a44519a4.slice/crio-3bd13d2dbe839c78bb3ab1666af14d7b573856bc1caf39fb397e7eefd8883351 WatchSource:0}: Error finding container 3bd13d2dbe839c78bb3ab1666af14d7b573856bc1caf39fb397e7eefd8883351: Status 404 returned error can't find the container with id 3bd13d2dbe839c78bb3ab1666af14d7b573856bc1caf39fb397e7eefd8883351 Sep 07 09:00:22.353676 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:22.353643 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c97495d-c67c9" event={"ID":"210f99b4-6ff3-40ee-acdd-de81a44519a4","Type":"ContainerStarted","Data":"3bd13d2dbe839c78bb3ab1666af14d7b573856bc1caf39fb397e7eefd8883351"} Sep 07 09:00:23.361404 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:23.361323 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-7dqmf" event={"ID":"fdd66e31-1fc9-41c2-8c70-d5dd7c29d836","Type":"ContainerStarted","Data":"1ce339c4894dedc9c47ddbb42e61c57faccc8fa21c76d19304f2844dbb04a632"} Sep 07 09:00:23.362137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:23.362051 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-7dqmf" Sep 07 09:00:23.372607 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:23.372577 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-7dqmf" Sep 07 09:00:23.384835 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:23.384782 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-7dqmf" podStartSLOduration=2.007305888 podStartE2EDuration="20.384767334s" totalImagesPullingTime="18.377461446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:23.383226317 +0000 UTC m=+81.900017233" watchObservedRunningTime="2026-09-07 09:00:23.384767334 +0000 UTC m=+81.901558236" Sep 07 09:00:24.553965 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.553815 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c6c5dc787-4cx6m"] Sep 07 09:00:24.563892 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.563861 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.573554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.573495 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6c5dc787-4cx6m"] Sep 07 09:00:24.574996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.574963 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 07 09:00:24.692156 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.692115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-oauth-serving-cert\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.692353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.692163 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-oauth-config\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.692353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.692209 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-service-ca\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.692353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.692264 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-trusted-ca-bundle\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.692353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.692334 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-config\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.692557 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.692362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8xcr\" (UniqueName: \"kubernetes.io/projected/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-kube-api-access-x8xcr\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.692557 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.692396 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-serving-cert\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.793403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.793362 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-serving-cert\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.793599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.793453 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-oauth-serving-cert\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.793599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.793486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-oauth-config\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.793599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.793528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-service-ca\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.793599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.793551 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-trusted-ca-bundle\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.793599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.793581 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-config\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.793857 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.793606 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8xcr\" (UniqueName: \"kubernetes.io/projected/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-kube-api-access-x8xcr\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.794807 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.794777 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-service-ca\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.795072 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.795044 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-oauth-serving-cert\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.795335 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.795293 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-config\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.795430 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.795355 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-trusted-ca-bundle\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.797493 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.797359 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-serving-cert\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.797493 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.797455 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-oauth-config\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.805552 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.805478 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8xcr\" (UniqueName: \"kubernetes.io/projected/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-kube-api-access-x8xcr\") pod \"console-7c6c5dc787-4cx6m\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:24.881774 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:24.881493 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:26.049569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:26.049276 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6c5dc787-4cx6m"] Sep 07 09:00:26.115769 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:00:26.115741 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97dbc82d_5a12_46b4_a01a_c4e4ff82ccc7.slice/crio-1c29a109e7795f0097d0e8513939de7bc3f7fbc4c0ee6d1f6972cd02457b76d8 WatchSource:0}: Error finding container 1c29a109e7795f0097d0e8513939de7bc3f7fbc4c0ee6d1f6972cd02457b76d8: Status 404 returned error can't find the container with id 1c29a109e7795f0097d0e8513939de7bc3f7fbc4c0ee6d1f6972cd02457b76d8 Sep 07 09:00:26.377202 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:26.376885 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c97495d-c67c9" event={"ID":"210f99b4-6ff3-40ee-acdd-de81a44519a4","Type":"ContainerStarted","Data":"90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03"} Sep 07 09:00:26.378989 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:26.378957 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c5dc787-4cx6m" event={"ID":"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7","Type":"ContainerStarted","Data":"868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7"} Sep 07 09:00:26.379125 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:26.379010 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c5dc787-4cx6m" event={"ID":"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7","Type":"ContainerStarted","Data":"1c29a109e7795f0097d0e8513939de7bc3f7fbc4c0ee6d1f6972cd02457b76d8"} Sep 07 09:00:26.409741 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:26.409688 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-576c97495d-c67c9" podStartSLOduration=8.522119398 podStartE2EDuration="12.409667945s" totalImagesPullingTime="3.887548547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:26.408208156 +0000 UTC m=+84.924999072" watchObservedRunningTime="2026-09-07 09:00:26.409667945 +0000 UTC m=+84.926458859" Sep 07 09:00:26.440329 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:26.440257 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c6c5dc787-4cx6m" podStartSLOduration=2.440236627 podStartE2EDuration="2.440236627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:26.43982875 +0000 UTC m=+84.956619655" watchObservedRunningTime="2026-09-07 09:00:26.440236627 +0000 UTC m=+84.957027535" Sep 07 09:00:34.882613 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:34.882579 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:34.883067 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:34.882661 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:34.887422 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:34.887398 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:35.015692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:35.015665 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:35.015957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:35.015927 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:35.020314 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:35.020275 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:35.421031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:35.421002 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:00:35.421262 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:35.421060 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:00:35.483165 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:35.483134 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576c97495d-c67c9"] Sep 07 09:00:42.896262 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:42.896229 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-582vt_cde1e62f-e104-437f-82ad-fd68be33a484/serve-healthcheck-canary/0.log" Sep 07 09:00:45.327003 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:00:45.326976 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kl8g9" Sep 07 09:01:02.442762 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.442687 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-576c97495d-c67c9" podUID="210f99b4-6ff3-40ee-acdd-de81a44519a4" containerName="console" containerID="cri-o://90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03" gracePeriod=15 Sep 07 09:01:02.716958 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.716901 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:01:02.871246 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871215 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-oauth-config\") pod \"210f99b4-6ff3-40ee-acdd-de81a44519a4\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " Sep 07 09:01:02.871246 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871256 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-service-ca\") pod \"210f99b4-6ff3-40ee-acdd-de81a44519a4\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " Sep 07 09:01:02.871490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871327 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-oauth-serving-cert\") pod \"210f99b4-6ff3-40ee-acdd-de81a44519a4\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " Sep 07 09:01:02.871490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871353 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-config\") pod \"210f99b4-6ff3-40ee-acdd-de81a44519a4\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " Sep 07 09:01:02.871490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871394 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-serving-cert\") pod \"210f99b4-6ff3-40ee-acdd-de81a44519a4\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " Sep 07 09:01:02.871490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871412 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq854\" (UniqueName: \"kubernetes.io/projected/210f99b4-6ff3-40ee-acdd-de81a44519a4-kube-api-access-nq854\") pod \"210f99b4-6ff3-40ee-acdd-de81a44519a4\" (UID: \"210f99b4-6ff3-40ee-acdd-de81a44519a4\") " Sep 07 09:01:02.871702 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871656 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-service-ca" (OuterVolumeSpecName: "service-ca") pod "210f99b4-6ff3-40ee-acdd-de81a44519a4" (UID: "210f99b4-6ff3-40ee-acdd-de81a44519a4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:01:02.871809 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871783 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "210f99b4-6ff3-40ee-acdd-de81a44519a4" (UID: "210f99b4-6ff3-40ee-acdd-de81a44519a4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:01:02.871910 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.871855 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-config" (OuterVolumeSpecName: "console-config") pod "210f99b4-6ff3-40ee-acdd-de81a44519a4" (UID: "210f99b4-6ff3-40ee-acdd-de81a44519a4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:01:02.873562 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.873534 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "210f99b4-6ff3-40ee-acdd-de81a44519a4" (UID: "210f99b4-6ff3-40ee-acdd-de81a44519a4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:01:02.873562 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.873546 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210f99b4-6ff3-40ee-acdd-de81a44519a4-kube-api-access-nq854" (OuterVolumeSpecName: "kube-api-access-nq854") pod "210f99b4-6ff3-40ee-acdd-de81a44519a4" (UID: "210f99b4-6ff3-40ee-acdd-de81a44519a4"). InnerVolumeSpecName "kube-api-access-nq854". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:01:02.873696 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.873558 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "210f99b4-6ff3-40ee-acdd-de81a44519a4" (UID: "210f99b4-6ff3-40ee-acdd-de81a44519a4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:01:02.972839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.972744 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-oauth-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:01:02.972839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.972778 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:01:02.972839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.972788 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:01:02.972839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.972797 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq854\" (UniqueName: \"kubernetes.io/projected/210f99b4-6ff3-40ee-acdd-de81a44519a4-kube-api-access-nq854\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:01:02.972839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.972805 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/210f99b4-6ff3-40ee-acdd-de81a44519a4-console-oauth-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:01:02.972839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:02.972813 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/210f99b4-6ff3-40ee-acdd-de81a44519a4-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:01:03.520938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.520900 2558 generic.go:356] "Generic (PLEG): container finished" podID="210f99b4-6ff3-40ee-acdd-de81a44519a4" containerID="90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03" exitCode=0 Sep 07 09:01:03.520938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.520939 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c97495d-c67c9" Sep 07 09:01:03.521466 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.520984 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c97495d-c67c9" event={"ID":"210f99b4-6ff3-40ee-acdd-de81a44519a4","Type":"ContainerDied","Data":"90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03"} Sep 07 09:01:03.521466 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.521035 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c97495d-c67c9" event={"ID":"210f99b4-6ff3-40ee-acdd-de81a44519a4","Type":"ContainerDied","Data":"3bd13d2dbe839c78bb3ab1666af14d7b573856bc1caf39fb397e7eefd8883351"} Sep 07 09:01:03.521466 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.521064 2558 scope.go:117] "RemoveContainer" containerID="90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03" Sep 07 09:01:03.529282 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.529263 2558 scope.go:117] "RemoveContainer" containerID="90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03" Sep 07 09:01:03.529582 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:01:03.529562 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03\": container with ID starting with 90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03 not found: ID does not exist" containerID="90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03" Sep 07 09:01:03.529650 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.529594 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03"} err="failed to get container status \"90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03\": rpc error: code = NotFound desc = could not find container \"90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03\": container with ID starting with 90417aa41c4f91f6f813c5db6bbde95eebf74eced661f15a98096b90d5cdaf03 not found: ID does not exist" Sep 07 09:01:03.539860 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.539840 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576c97495d-c67c9"] Sep 07 09:01:03.543682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.543664 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-576c97495d-c67c9"] Sep 07 09:01:03.950310 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:03.950217 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="210f99b4-6ff3-40ee-acdd-de81a44519a4" path="/var/lib/kubelet/pods/210f99b4-6ff3-40ee-acdd-de81a44519a4/volumes" Sep 07 09:01:27.420252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.420214 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54f966fffb-9mh48"] Sep 07 09:01:27.420807 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.420553 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="210f99b4-6ff3-40ee-acdd-de81a44519a4" containerName="console" Sep 07 09:01:27.420807 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.420566 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="210f99b4-6ff3-40ee-acdd-de81a44519a4" containerName="console" Sep 07 09:01:27.420807 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.420610 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="210f99b4-6ff3-40ee-acdd-de81a44519a4" containerName="console" Sep 07 09:01:27.423559 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.423543 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.427502 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.427476 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 07 09:01:27.427502 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.427476 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 07 09:01:27.427682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.427522 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 07 09:01:27.427682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.427499 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 07 09:01:27.427682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.427605 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 07 09:01:27.427682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.427605 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-92rml\"" Sep 07 09:01:27.431231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431202 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-serving-certs-ca-bundle\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.431690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431253 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-metrics-client-ca\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.431690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431321 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-secret-telemeter-client\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.431690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62qjf\" (UniqueName: \"kubernetes.io/projected/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-kube-api-access-62qjf\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.431690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431392 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-federate-client-tls\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.431690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431462 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.431690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431505 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.431690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.431545 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-telemeter-client-tls\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.432423 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.432405 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 07 09:01:27.435286 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.435264 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f966fffb-9mh48"] Sep 07 09:01:27.532617 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532580 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-telemeter-client-tls\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.532617 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-serving-certs-ca-bundle\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.532859 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532649 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-metrics-client-ca\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.532859 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532678 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-secret-telemeter-client\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.532859 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532699 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62qjf\" (UniqueName: \"kubernetes.io/projected/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-kube-api-access-62qjf\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.532859 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532724 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-federate-client-tls\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.532859 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532777 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.532859 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.532817 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.533497 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.533467 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-serving-certs-ca-bundle\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.533683 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.533601 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-metrics-client-ca\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.533765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.533677 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.535251 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.535227 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.535629 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.535607 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-telemeter-client-tls\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.535736 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.535718 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-federate-client-tls\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.536009 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.535988 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-secret-telemeter-client\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.542883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.542859 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62qjf\" (UniqueName: \"kubernetes.io/projected/0f883935-c1c4-48fc-b1ce-048ceb8ef3bf-kube-api-access-62qjf\") pod \"telemeter-client-54f966fffb-9mh48\" (UID: \"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf\") " pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:27.736119 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:27.736001 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" Sep 07 09:01:28.025103 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:28.024095 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f966fffb-9mh48"] Sep 07 09:01:28.026844 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:01:28.026814 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f883935_c1c4_48fc_b1ce_048ceb8ef3bf.slice/crio-2eb3f6c76fabb088adc730b269623100b91018bd7191524b6cc2827b66971997 WatchSource:0}: Error finding container 2eb3f6c76fabb088adc730b269623100b91018bd7191524b6cc2827b66971997: Status 404 returned error can't find the container with id 2eb3f6c76fabb088adc730b269623100b91018bd7191524b6cc2827b66971997 Sep 07 09:01:28.611471 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:28.611436 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" event={"ID":"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf","Type":"ContainerStarted","Data":"2eb3f6c76fabb088adc730b269623100b91018bd7191524b6cc2827b66971997"} Sep 07 09:01:30.622028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:30.621995 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" event={"ID":"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf","Type":"ContainerStarted","Data":"348601f47e519c3508c18793da58db2a3c2affc3ede5e43f2dec42fe114d4b48"} Sep 07 09:01:32.632269 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.632236 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" event={"ID":"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf","Type":"ContainerStarted","Data":"bbd35b30381e439088832c74e26c7db0af52033f37fecaf6f349986c1f1a37bd"} Sep 07 09:01:32.632693 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.632306 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" event={"ID":"0f883935-c1c4-48fc-b1ce-048ceb8ef3bf","Type":"ContainerStarted","Data":"f4a5858af128ac8687ebaf10bc0b69527a8bc4716d99c07e90f04096fb9bda22"} Sep 07 09:01:32.654644 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.654596 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54f966fffb-9mh48" podStartSLOduration=1.871450298 podStartE2EDuration="5.654580675s" totalImagesPullingTime="3.783130377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:01:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:01:32.652682576 +0000 UTC m=+151.169473478" watchObservedRunningTime="2026-09-07 09:01:32.654580675 +0000 UTC m=+151.171371579" Sep 07 09:01:32.737674 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.737643 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f468b57c5-6d2lq"] Sep 07 09:01:32.740501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.740484 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.753200 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.753178 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f468b57c5-6d2lq"] Sep 07 09:01:32.772815 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.772789 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-console-config\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.773133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.772862 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hslzr\" (UniqueName: \"kubernetes.io/projected/cb4686f2-5e9c-4618-a44b-170baa764eba-kube-api-access-hslzr\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.773133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.772904 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-serving-cert\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.773133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.772984 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-oauth-serving-cert\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.773133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.773035 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-service-ca\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.773133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.773083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-trusted-ca-bundle\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.773133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.773128 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-oauth-config\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.874465 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.874432 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-oauth-config\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.874609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.874479 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-console-config\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.874609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.874496 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hslzr\" (UniqueName: \"kubernetes.io/projected/cb4686f2-5e9c-4618-a44b-170baa764eba-kube-api-access-hslzr\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.874609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.874521 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-serving-cert\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.874609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.874556 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-oauth-serving-cert\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.874609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.874578 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-service-ca\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.874867 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.874613 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-trusted-ca-bundle\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.875403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.875377 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-oauth-serving-cert\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.875511 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.875381 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-service-ca\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.875511 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.875431 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-console-config\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.875511 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.875491 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-trusted-ca-bundle\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.876938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.876916 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-oauth-config\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.877104 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.877086 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-serving-cert\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:32.883499 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:32.883443 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hslzr\" (UniqueName: \"kubernetes.io/projected/cb4686f2-5e9c-4618-a44b-170baa764eba-kube-api-access-hslzr\") pod \"console-6f468b57c5-6d2lq\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:33.048523 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:33.048491 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:33.320333 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:33.320284 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f468b57c5-6d2lq"] Sep 07 09:01:33.323363 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:01:33.323339 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4686f2_5e9c_4618_a44b_170baa764eba.slice/crio-9c871b8cb78aa96acd136192c9100cbf23ef775ecfdabefaa26059bd84d4de79 WatchSource:0}: Error finding container 9c871b8cb78aa96acd136192c9100cbf23ef775ecfdabefaa26059bd84d4de79: Status 404 returned error can't find the container with id 9c871b8cb78aa96acd136192c9100cbf23ef775ecfdabefaa26059bd84d4de79 Sep 07 09:01:33.641600 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:33.641500 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f468b57c5-6d2lq" event={"ID":"cb4686f2-5e9c-4618-a44b-170baa764eba","Type":"ContainerStarted","Data":"87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765"} Sep 07 09:01:33.641600 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:33.641593 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f468b57c5-6d2lq" event={"ID":"cb4686f2-5e9c-4618-a44b-170baa764eba","Type":"ContainerStarted","Data":"9c871b8cb78aa96acd136192c9100cbf23ef775ecfdabefaa26059bd84d4de79"} Sep 07 09:01:33.660312 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:33.660227 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f468b57c5-6d2lq" podStartSLOduration=1.660205908 podStartE2EDuration="1.660205908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:01:33.658930438 +0000 UTC m=+152.175721341" watchObservedRunningTime="2026-09-07 09:01:33.660205908 +0000 UTC m=+152.176996814" Sep 07 09:01:43.048721 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:43.048681 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:43.049320 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:43.048755 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:43.053566 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:43.053542 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:43.682450 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:43.682425 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:01:43.726836 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:01:43.726807 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6c5dc787-4cx6m"] Sep 07 09:02:08.746748 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:08.746705 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c6c5dc787-4cx6m" podUID="97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" containerName="console" containerID="cri-o://868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7" gracePeriod=15 Sep 07 09:02:08.989086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:08.989061 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:02:09.143001 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.142974 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-serving-cert\") pod \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " Sep 07 09:02:09.143179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143012 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-service-ca\") pod \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " Sep 07 09:02:09.143179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143036 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-trusted-ca-bundle\") pod \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " Sep 07 09:02:09.143179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143058 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-oauth-config\") pod \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " Sep 07 09:02:09.143179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143090 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-oauth-serving-cert\") pod \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " Sep 07 09:02:09.143179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143117 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-config\") pod \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " Sep 07 09:02:09.143179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143143 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8xcr\" (UniqueName: \"kubernetes.io/projected/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-kube-api-access-x8xcr\") pod \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\" (UID: \"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7\") " Sep 07 09:02:09.143495 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143353 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-service-ca" (OuterVolumeSpecName: "service-ca") pod "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" (UID: "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:02:09.143495 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143404 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" (UID: "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:02:09.143847 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143824 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" (UID: "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:02:09.143946 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.143841 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-config" (OuterVolumeSpecName: "console-config") pod "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" (UID: "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:02:09.145248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.145217 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" (UID: "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:02:09.145248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.145227 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-kube-api-access-x8xcr" (OuterVolumeSpecName: "kube-api-access-x8xcr") pod "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" (UID: "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7"). InnerVolumeSpecName "kube-api-access-x8xcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:02:09.145531 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.145505 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" (UID: "97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:02:09.244439 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.244391 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:02:09.244439 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.244433 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8xcr\" (UniqueName: \"kubernetes.io/projected/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-kube-api-access-x8xcr\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:02:09.244439 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.244443 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:02:09.244439 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.244453 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:02:09.244675 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.244461 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-trusted-ca-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:02:09.244675 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.244470 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-console-oauth-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:02:09.244675 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.244478 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7-oauth-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:02:09.773550 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.773514 2558 generic.go:356] "Generic (PLEG): container finished" podID="97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" containerID="868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7" exitCode=0 Sep 07 09:02:09.774004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.773553 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c5dc787-4cx6m" event={"ID":"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7","Type":"ContainerDied","Data":"868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7"} Sep 07 09:02:09.774004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.773588 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6c5dc787-4cx6m" Sep 07 09:02:09.774004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.773611 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6c5dc787-4cx6m" event={"ID":"97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7","Type":"ContainerDied","Data":"1c29a109e7795f0097d0e8513939de7bc3f7fbc4c0ee6d1f6972cd02457b76d8"} Sep 07 09:02:09.774004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.773634 2558 scope.go:117] "RemoveContainer" containerID="868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7" Sep 07 09:02:09.782122 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.782107 2558 scope.go:117] "RemoveContainer" containerID="868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7" Sep 07 09:02:09.782414 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:02:09.782396 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7\": container with ID starting with 868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7 not found: ID does not exist" containerID="868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7" Sep 07 09:02:09.782475 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.782423 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7"} err="failed to get container status \"868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7\": rpc error: code = NotFound desc = could not find container \"868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7\": container with ID starting with 868654c3d849c4127a85bb8b32186d011a7c9bf6b29e85b406e821db970524a7 not found: ID does not exist" Sep 07 09:02:09.793247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.793228 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6c5dc787-4cx6m"] Sep 07 09:02:09.797460 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.797440 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c6c5dc787-4cx6m"] Sep 07 09:02:09.949474 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:09.949441 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" path="/var/lib/kubelet/pods/97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7/volumes" Sep 07 09:02:41.372479 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.372445 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67546f76b9-5j685"] Sep 07 09:02:41.372891 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.372702 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" containerName="console" Sep 07 09:02:41.372891 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.372713 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" containerName="console" Sep 07 09:02:41.372891 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.372763 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="97dbc82d-5a12-46b4-a01a-c4e4ff82ccc7" containerName="console" Sep 07 09:02:41.375621 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.375604 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.385581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.385551 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67546f76b9-5j685"] Sep 07 09:02:41.481384 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.481349 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-console-config\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.481384 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.481389 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-oauth-serving-cert\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.481591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.481418 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcbph\" (UniqueName: \"kubernetes.io/projected/7f9d2657-61a9-4149-93d1-4373effdc096-kube-api-access-vcbph\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.481591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.481496 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-trusted-ca-bundle\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.481591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.481529 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-oauth-config\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.481591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.481565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-serving-cert\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.481591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.481586 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-service-ca\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.582690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.582661 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-oauth-config\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.582759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.582708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-serving-cert\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.582759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.582731 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-service-ca\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.582833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.582757 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-console-config\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.582833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.582779 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-oauth-serving-cert\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.582833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.582805 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcbph\" (UniqueName: \"kubernetes.io/projected/7f9d2657-61a9-4149-93d1-4373effdc096-kube-api-access-vcbph\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.582973 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.582833 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-trusted-ca-bundle\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.583650 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.583621 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-service-ca\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.583788 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.583621 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-oauth-serving-cert\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.583788 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.583694 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-console-config\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.583788 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.583765 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-trusted-ca-bundle\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.585137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.585111 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-serving-cert\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.585252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.585216 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-oauth-config\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.590894 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.590871 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcbph\" (UniqueName: \"kubernetes.io/projected/7f9d2657-61a9-4149-93d1-4373effdc096-kube-api-access-vcbph\") pod \"console-67546f76b9-5j685\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.684120 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.684028 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:41.954568 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:41.954458 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67546f76b9-5j685"] Sep 07 09:02:41.957129 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:02:41.957104 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f9d2657_61a9_4149_93d1_4373effdc096.slice/crio-de1f6ff1024f4d1d60f0d9dacae021ec6aea612d553e8f1d4464e1ead31b4f64 WatchSource:0}: Error finding container de1f6ff1024f4d1d60f0d9dacae021ec6aea612d553e8f1d4464e1ead31b4f64: Status 404 returned error can't find the container with id de1f6ff1024f4d1d60f0d9dacae021ec6aea612d553e8f1d4464e1ead31b4f64 Sep 07 09:02:42.891595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:42.891553 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67546f76b9-5j685" event={"ID":"7f9d2657-61a9-4149-93d1-4373effdc096","Type":"ContainerStarted","Data":"abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417"} Sep 07 09:02:42.891979 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:42.891617 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67546f76b9-5j685" event={"ID":"7f9d2657-61a9-4149-93d1-4373effdc096","Type":"ContainerStarted","Data":"de1f6ff1024f4d1d60f0d9dacae021ec6aea612d553e8f1d4464e1ead31b4f64"} Sep 07 09:02:42.909857 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:42.909809 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67546f76b9-5j685" podStartSLOduration=1.909793493 podStartE2EDuration="1.909793493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:02:42.908042975 +0000 UTC m=+221.424833895" watchObservedRunningTime="2026-09-07 09:02:42.909793493 +0000 UTC m=+221.426584395" Sep 07 09:02:51.684509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:51.684450 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:51.684509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:51.684526 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:51.689391 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:51.689366 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:51.929704 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:51.929673 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:02:51.976948 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:02:51.976865 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f468b57c5-6d2lq"] Sep 07 09:03:16.998844 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:16.998784 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f468b57c5-6d2lq" podUID="cb4686f2-5e9c-4618-a44b-170baa764eba" containerName="console" containerID="cri-o://87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765" gracePeriod=15 Sep 07 09:03:17.243183 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.243161 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:03:17.340540 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340507 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-oauth-serving-cert\") pod \"cb4686f2-5e9c-4618-a44b-170baa764eba\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " Sep 07 09:03:17.340718 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340560 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-trusted-ca-bundle\") pod \"cb4686f2-5e9c-4618-a44b-170baa764eba\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " Sep 07 09:03:17.340718 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340590 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-serving-cert\") pod \"cb4686f2-5e9c-4618-a44b-170baa764eba\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " Sep 07 09:03:17.340718 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340613 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-service-ca\") pod \"cb4686f2-5e9c-4618-a44b-170baa764eba\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " Sep 07 09:03:17.340718 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340647 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-oauth-config\") pod \"cb4686f2-5e9c-4618-a44b-170baa764eba\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " Sep 07 09:03:17.340933 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340783 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-console-config\") pod \"cb4686f2-5e9c-4618-a44b-170baa764eba\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " Sep 07 09:03:17.340933 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340839 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hslzr\" (UniqueName: \"kubernetes.io/projected/cb4686f2-5e9c-4618-a44b-170baa764eba-kube-api-access-hslzr\") pod \"cb4686f2-5e9c-4618-a44b-170baa764eba\" (UID: \"cb4686f2-5e9c-4618-a44b-170baa764eba\") " Sep 07 09:03:17.341026 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.340991 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb4686f2-5e9c-4618-a44b-170baa764eba" (UID: "cb4686f2-5e9c-4618-a44b-170baa764eba"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:03:17.341078 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.341067 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-oauth-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:17.341129 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.341113 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-console-config" (OuterVolumeSpecName: "console-config") pod "cb4686f2-5e9c-4618-a44b-170baa764eba" (UID: "cb4686f2-5e9c-4618-a44b-170baa764eba"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:03:17.341183 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.341139 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb4686f2-5e9c-4618-a44b-170baa764eba" (UID: "cb4686f2-5e9c-4618-a44b-170baa764eba"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:03:17.341183 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.341156 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cb4686f2-5e9c-4618-a44b-170baa764eba" (UID: "cb4686f2-5e9c-4618-a44b-170baa764eba"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:03:17.342791 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.342768 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb4686f2-5e9c-4618-a44b-170baa764eba" (UID: "cb4686f2-5e9c-4618-a44b-170baa764eba"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:03:17.342887 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.342830 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb4686f2-5e9c-4618-a44b-170baa764eba" (UID: "cb4686f2-5e9c-4618-a44b-170baa764eba"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:03:17.342940 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.342901 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb4686f2-5e9c-4618-a44b-170baa764eba-kube-api-access-hslzr" (OuterVolumeSpecName: "kube-api-access-hslzr") pod "cb4686f2-5e9c-4618-a44b-170baa764eba" (UID: "cb4686f2-5e9c-4618-a44b-170baa764eba"). InnerVolumeSpecName "kube-api-access-hslzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:03:17.442383 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.442352 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:17.442383 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.442379 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:17.442383 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.442388 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb4686f2-5e9c-4618-a44b-170baa764eba-console-oauth-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:17.442595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.442396 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-console-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:17.442595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.442405 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hslzr\" (UniqueName: \"kubernetes.io/projected/cb4686f2-5e9c-4618-a44b-170baa764eba-kube-api-access-hslzr\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:17.442595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:17.442413 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb4686f2-5e9c-4618-a44b-170baa764eba-trusted-ca-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:18.018081 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.018049 2558 generic.go:356] "Generic (PLEG): container finished" podID="cb4686f2-5e9c-4618-a44b-170baa764eba" containerID="87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765" exitCode=0 Sep 07 09:03:18.018495 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.018090 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f468b57c5-6d2lq" Sep 07 09:03:18.018495 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.018135 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f468b57c5-6d2lq" event={"ID":"cb4686f2-5e9c-4618-a44b-170baa764eba","Type":"ContainerDied","Data":"87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765"} Sep 07 09:03:18.018495 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.018185 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f468b57c5-6d2lq" event={"ID":"cb4686f2-5e9c-4618-a44b-170baa764eba","Type":"ContainerDied","Data":"9c871b8cb78aa96acd136192c9100cbf23ef775ecfdabefaa26059bd84d4de79"} Sep 07 09:03:18.018495 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.018205 2558 scope.go:117] "RemoveContainer" containerID="87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765" Sep 07 09:03:18.025864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.025838 2558 scope.go:117] "RemoveContainer" containerID="87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765" Sep 07 09:03:18.026124 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:03:18.026106 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765\": container with ID starting with 87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765 not found: ID does not exist" containerID="87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765" Sep 07 09:03:18.026172 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.026133 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765"} err="failed to get container status \"87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765\": rpc error: code = NotFound desc = could not find container \"87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765\": container with ID starting with 87f3e6051daaaea6669e5893dc7c7ce82a32b87b2bbe64a73c76d63741e1e765 not found: ID does not exist" Sep 07 09:03:18.033249 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.033229 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f468b57c5-6d2lq"] Sep 07 09:03:18.039152 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:18.039131 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f468b57c5-6d2lq"] Sep 07 09:03:19.950040 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:19.950005 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb4686f2-5e9c-4618-a44b-170baa764eba" path="/var/lib/kubelet/pods/cb4686f2-5e9c-4618-a44b-170baa764eba/volumes" Sep 07 09:03:25.592568 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.592489 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q"] Sep 07 09:03:25.592912 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.592786 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb4686f2-5e9c-4618-a44b-170baa764eba" containerName="console" Sep 07 09:03:25.592912 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.592799 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb4686f2-5e9c-4618-a44b-170baa764eba" containerName="console" Sep 07 09:03:25.592912 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.592857 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb4686f2-5e9c-4618-a44b-170baa764eba" containerName="console" Sep 07 09:03:25.595681 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.595663 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.598024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.598004 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 07 09:03:25.598944 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.598930 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f5tkk\"" Sep 07 09:03:25.599019 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.598955 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 07 09:03:25.604435 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.604416 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q"] Sep 07 09:03:25.703605 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.703573 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/8037179b-c200-45d6-a1dd-d792dd02610b-kube-api-access-nl2x8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.703785 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.703614 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.703785 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.703660 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.804912 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.804882 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/8037179b-c200-45d6-a1dd-d792dd02610b-kube-api-access-nl2x8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.805055 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.804920 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.805055 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.804965 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.805283 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.805267 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.805342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.805318 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.813021 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.813001 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/8037179b-c200-45d6-a1dd-d792dd02610b-kube-api-access-nl2x8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:25.905194 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:25.905113 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:26.173566 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:26.173154 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q"] Sep 07 09:03:26.180240 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:03:26.180211 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8037179b_c200_45d6_a1dd_d792dd02610b.slice/crio-7d074c86fdceef80fa2719b9c77167c539d2864beab620b5112a17a40bab1238 WatchSource:0}: Error finding container 7d074c86fdceef80fa2719b9c77167c539d2864beab620b5112a17a40bab1238: Status 404 returned error can't find the container with id 7d074c86fdceef80fa2719b9c77167c539d2864beab620b5112a17a40bab1238 Sep 07 09:03:27.052537 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:27.052490 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" event={"ID":"8037179b-c200-45d6-a1dd-d792dd02610b","Type":"ContainerStarted","Data":"7d074c86fdceef80fa2719b9c77167c539d2864beab620b5112a17a40bab1238"} Sep 07 09:03:32.075672 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:32.075640 2558 generic.go:356] "Generic (PLEG): container finished" podID="8037179b-c200-45d6-a1dd-d792dd02610b" containerID="de61df22330e1d8488e380a18b89a92fa3269f0979ec774863d437a155105272" exitCode=0 Sep 07 09:03:32.076023 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:32.075684 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" event={"ID":"8037179b-c200-45d6-a1dd-d792dd02610b","Type":"ContainerDied","Data":"de61df22330e1d8488e380a18b89a92fa3269f0979ec774863d437a155105272"} Sep 07 09:03:35.090879 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:35.090842 2558 generic.go:356] "Generic (PLEG): container finished" podID="8037179b-c200-45d6-a1dd-d792dd02610b" containerID="75df11c8b9f6a3cd93c275bdfd35f8d1939c78c7e38ee9235bbf544f051edb94" exitCode=0 Sep 07 09:03:35.091240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:35.090934 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" event={"ID":"8037179b-c200-45d6-a1dd-d792dd02610b","Type":"ContainerDied","Data":"75df11c8b9f6a3cd93c275bdfd35f8d1939c78c7e38ee9235bbf544f051edb94"} Sep 07 09:03:42.119189 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:42.119149 2558 generic.go:356] "Generic (PLEG): container finished" podID="8037179b-c200-45d6-a1dd-d792dd02610b" containerID="e825e08181656ad4abbe6c3addd9c80fae027cce15cdca313c4a40a505be1a67" exitCode=0 Sep 07 09:03:42.119574 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:42.119230 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" event={"ID":"8037179b-c200-45d6-a1dd-d792dd02610b","Type":"ContainerDied","Data":"e825e08181656ad4abbe6c3addd9c80fae027cce15cdca313c4a40a505be1a67"} Sep 07 09:03:43.241084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.241059 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:43.356861 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.356761 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/8037179b-c200-45d6-a1dd-d792dd02610b-kube-api-access-nl2x8\") pod \"8037179b-c200-45d6-a1dd-d792dd02610b\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " Sep 07 09:03:43.357020 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.356884 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-bundle\") pod \"8037179b-c200-45d6-a1dd-d792dd02610b\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " Sep 07 09:03:43.357020 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.356922 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-util\") pod \"8037179b-c200-45d6-a1dd-d792dd02610b\" (UID: \"8037179b-c200-45d6-a1dd-d792dd02610b\") " Sep 07 09:03:43.357595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.357569 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-bundle" (OuterVolumeSpecName: "bundle") pod "8037179b-c200-45d6-a1dd-d792dd02610b" (UID: "8037179b-c200-45d6-a1dd-d792dd02610b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:03:43.358924 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.358892 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8037179b-c200-45d6-a1dd-d792dd02610b-kube-api-access-nl2x8" (OuterVolumeSpecName: "kube-api-access-nl2x8") pod "8037179b-c200-45d6-a1dd-d792dd02610b" (UID: "8037179b-c200-45d6-a1dd-d792dd02610b"). InnerVolumeSpecName "kube-api-access-nl2x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:03:43.361584 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.361547 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-util" (OuterVolumeSpecName: "util") pod "8037179b-c200-45d6-a1dd-d792dd02610b" (UID: "8037179b-c200-45d6-a1dd-d792dd02610b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:03:43.458231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.458135 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nl2x8\" (UniqueName: \"kubernetes.io/projected/8037179b-c200-45d6-a1dd-d792dd02610b-kube-api-access-nl2x8\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:43.458231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.458177 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:43.458231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:43.458188 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8037179b-c200-45d6-a1dd-d792dd02610b-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:03:44.127197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:44.127169 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" Sep 07 09:03:44.127197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:44.127183 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754046t6q" event={"ID":"8037179b-c200-45d6-a1dd-d792dd02610b","Type":"ContainerDied","Data":"7d074c86fdceef80fa2719b9c77167c539d2864beab620b5112a17a40bab1238"} Sep 07 09:03:44.127429 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:44.127232 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d074c86fdceef80fa2719b9c77167c539d2864beab620b5112a17a40bab1238" Sep 07 09:03:48.312840 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.312809 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4"] Sep 07 09:03:48.313259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.313065 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8037179b-c200-45d6-a1dd-d792dd02610b" containerName="extract" Sep 07 09:03:48.313259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.313075 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8037179b-c200-45d6-a1dd-d792dd02610b" containerName="extract" Sep 07 09:03:48.313259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.313085 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8037179b-c200-45d6-a1dd-d792dd02610b" containerName="pull" Sep 07 09:03:48.313259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.313090 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8037179b-c200-45d6-a1dd-d792dd02610b" containerName="pull" Sep 07 09:03:48.313259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.313108 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8037179b-c200-45d6-a1dd-d792dd02610b" containerName="util" Sep 07 09:03:48.313259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.313113 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8037179b-c200-45d6-a1dd-d792dd02610b" containerName="util" Sep 07 09:03:48.313259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.313156 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8037179b-c200-45d6-a1dd-d792dd02610b" containerName="extract" Sep 07 09:03:48.343145 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.343117 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4"] Sep 07 09:03:48.343320 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.343207 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.346472 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.346452 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 07 09:03:48.346866 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.346848 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 07 09:03:48.347097 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.347080 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-swph8\"" Sep 07 09:03:48.499142 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.499107 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e24bbab-da72-4590-ab24-e7da8bb59129-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zzhs4\" (UID: \"3e24bbab-da72-4590-ab24-e7da8bb59129\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.499381 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.499170 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpmxx\" (UniqueName: \"kubernetes.io/projected/3e24bbab-da72-4590-ab24-e7da8bb59129-kube-api-access-kpmxx\") pod \"cert-manager-operator-controller-manager-55c444bf86-zzhs4\" (UID: \"3e24bbab-da72-4590-ab24-e7da8bb59129\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.600110 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.600024 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpmxx\" (UniqueName: \"kubernetes.io/projected/3e24bbab-da72-4590-ab24-e7da8bb59129-kube-api-access-kpmxx\") pod \"cert-manager-operator-controller-manager-55c444bf86-zzhs4\" (UID: \"3e24bbab-da72-4590-ab24-e7da8bb59129\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.600392 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.600156 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e24bbab-da72-4590-ab24-e7da8bb59129-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zzhs4\" (UID: \"3e24bbab-da72-4590-ab24-e7da8bb59129\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.600638 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.600616 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e24bbab-da72-4590-ab24-e7da8bb59129-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zzhs4\" (UID: \"3e24bbab-da72-4590-ab24-e7da8bb59129\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.611462 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.611434 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpmxx\" (UniqueName: \"kubernetes.io/projected/3e24bbab-da72-4590-ab24-e7da8bb59129-kube-api-access-kpmxx\") pod \"cert-manager-operator-controller-manager-55c444bf86-zzhs4\" (UID: \"3e24bbab-da72-4590-ab24-e7da8bb59129\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.651614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.651581 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" Sep 07 09:03:48.953017 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:48.952948 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4"] Sep 07 09:03:48.956723 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:03:48.956694 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e24bbab_da72_4590_ab24_e7da8bb59129.slice/crio-6f9eaff1dd202b89ee31976af422a8a46833e493f601265b92ad0baecb1ae2f5 WatchSource:0}: Error finding container 6f9eaff1dd202b89ee31976af422a8a46833e493f601265b92ad0baecb1ae2f5: Status 404 returned error can't find the container with id 6f9eaff1dd202b89ee31976af422a8a46833e493f601265b92ad0baecb1ae2f5 Sep 07 09:03:49.147063 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:49.147026 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" event={"ID":"3e24bbab-da72-4590-ab24-e7da8bb59129","Type":"ContainerStarted","Data":"6f9eaff1dd202b89ee31976af422a8a46833e493f601265b92ad0baecb1ae2f5"} Sep 07 09:03:51.163450 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:51.163407 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" event={"ID":"3e24bbab-da72-4590-ab24-e7da8bb59129","Type":"ContainerStarted","Data":"7a941713358b25caecd61b19402577d8d1a4cc310422f8d253d6cc90a45d5cc9"} Sep 07 09:03:51.184425 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:51.184377 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zzhs4" podStartSLOduration=1.484261143 podStartE2EDuration="3.184362105s" totalImagesPullingTime="1.700100962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:03:51.182525508 +0000 UTC m=+289.699316414" watchObservedRunningTime="2026-09-07 09:03:51.184362105 +0000 UTC m=+289.701153009" Sep 07 09:03:52.588246 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.588204 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9lxh7"] Sep 07 09:03:52.591344 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.591328 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.593833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.593810 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-zfcrx\"" Sep 07 09:03:52.593833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.593820 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 07 09:03:52.594778 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.594758 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 07 09:03:52.600317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.600275 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9lxh7"] Sep 07 09:03:52.727872 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.727832 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbjj8\" (UniqueName: \"kubernetes.io/projected/fbad3ae7-36da-49af-b5f5-964c4ef47ad6-kube-api-access-hbjj8\") pod \"cert-manager-webhook-6486ffd997-9lxh7\" (UID: \"fbad3ae7-36da-49af-b5f5-964c4ef47ad6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.728047 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.727889 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbad3ae7-36da-49af-b5f5-964c4ef47ad6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9lxh7\" (UID: \"fbad3ae7-36da-49af-b5f5-964c4ef47ad6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.828411 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.828378 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbjj8\" (UniqueName: \"kubernetes.io/projected/fbad3ae7-36da-49af-b5f5-964c4ef47ad6-kube-api-access-hbjj8\") pod \"cert-manager-webhook-6486ffd997-9lxh7\" (UID: \"fbad3ae7-36da-49af-b5f5-964c4ef47ad6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.828571 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.828421 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbad3ae7-36da-49af-b5f5-964c4ef47ad6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9lxh7\" (UID: \"fbad3ae7-36da-49af-b5f5-964c4ef47ad6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.836146 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.836117 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbad3ae7-36da-49af-b5f5-964c4ef47ad6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9lxh7\" (UID: \"fbad3ae7-36da-49af-b5f5-964c4ef47ad6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.836259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.836229 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbjj8\" (UniqueName: \"kubernetes.io/projected/fbad3ae7-36da-49af-b5f5-964c4ef47ad6-kube-api-access-hbjj8\") pod \"cert-manager-webhook-6486ffd997-9lxh7\" (UID: \"fbad3ae7-36da-49af-b5f5-964c4ef47ad6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.917140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.917062 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:03:52.979123 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.979078 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899"] Sep 07 09:03:52.983722 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.983701 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:52.986250 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.986106 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 07 09:03:52.986250 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.986130 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 07 09:03:52.986537 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.986271 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f5tkk\"" Sep 07 09:03:52.989256 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:52.989233 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899"] Sep 07 09:03:53.130272 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.130235 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.130467 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.130293 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpstj\" (UniqueName: \"kubernetes.io/projected/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-kube-api-access-gpstj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.130467 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.130401 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.197000 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.196931 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9lxh7"] Sep 07 09:03:53.200437 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:03:53.200412 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbad3ae7_36da_49af_b5f5_964c4ef47ad6.slice/crio-b243a6ede5c0da24ae2924c8afe12f65466a2a2a9a2eb53c007a05a1052cec63 WatchSource:0}: Error finding container b243a6ede5c0da24ae2924c8afe12f65466a2a2a9a2eb53c007a05a1052cec63: Status 404 returned error can't find the container with id b243a6ede5c0da24ae2924c8afe12f65466a2a2a9a2eb53c007a05a1052cec63 Sep 07 09:03:53.231489 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.231466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.231600 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.231512 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.231600 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.231552 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpstj\" (UniqueName: \"kubernetes.io/projected/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-kube-api-access-gpstj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.231856 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.231835 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.231947 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.231928 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.240182 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.240155 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpstj\" (UniqueName: \"kubernetes.io/projected/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-kube-api-access-gpstj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.294626 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.294599 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:03:53.564525 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:53.564501 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899"] Sep 07 09:03:53.566826 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:03:53.566799 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0ccd0cf_ba50_4676_8d0d_740dd03e2dbc.slice/crio-c42e2340f89975e98265a65b060da98710e47f6d4bd3ffb412834bbf0cd3dac2 WatchSource:0}: Error finding container c42e2340f89975e98265a65b060da98710e47f6d4bd3ffb412834bbf0cd3dac2: Status 404 returned error can't find the container with id c42e2340f89975e98265a65b060da98710e47f6d4bd3ffb412834bbf0cd3dac2 Sep 07 09:03:54.175659 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.175623 2558 generic.go:356] "Generic (PLEG): container finished" podID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerID="be6d98d5735127c1fd6a63bb691977a7cc9e804d5e463c9e0eaa924474afcba9" exitCode=0 Sep 07 09:03:54.176114 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.175713 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" event={"ID":"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc","Type":"ContainerDied","Data":"be6d98d5735127c1fd6a63bb691977a7cc9e804d5e463c9e0eaa924474afcba9"} Sep 07 09:03:54.176114 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.175758 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" event={"ID":"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc","Type":"ContainerStarted","Data":"c42e2340f89975e98265a65b060da98710e47f6d4bd3ffb412834bbf0cd3dac2"} Sep 07 09:03:54.176817 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.176799 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" event={"ID":"fbad3ae7-36da-49af-b5f5-964c4ef47ad6","Type":"ContainerStarted","Data":"b243a6ede5c0da24ae2924c8afe12f65466a2a2a9a2eb53c007a05a1052cec63"} Sep 07 09:03:54.206584 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.206554 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb"] Sep 07 09:03:54.209610 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.209595 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.211818 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.211800 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dxbp8\"" Sep 07 09:03:54.219011 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.218992 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb"] Sep 07 09:03:54.341408 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.341376 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6jb5\" (UniqueName: \"kubernetes.io/projected/d4f0ec17-9ae0-4f7a-ade2-f976bf95df64-kube-api-access-g6jb5\") pod \"cert-manager-cainjector-559cdd8c87-tvhqb\" (UID: \"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.341550 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.341464 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4f0ec17-9ae0-4f7a-ade2-f976bf95df64-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tvhqb\" (UID: \"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.442525 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.442437 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6jb5\" (UniqueName: \"kubernetes.io/projected/d4f0ec17-9ae0-4f7a-ade2-f976bf95df64-kube-api-access-g6jb5\") pod \"cert-manager-cainjector-559cdd8c87-tvhqb\" (UID: \"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.442525 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.442482 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4f0ec17-9ae0-4f7a-ade2-f976bf95df64-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tvhqb\" (UID: \"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.450265 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.450243 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4f0ec17-9ae0-4f7a-ade2-f976bf95df64-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-tvhqb\" (UID: \"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.450466 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.450445 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6jb5\" (UniqueName: \"kubernetes.io/projected/d4f0ec17-9ae0-4f7a-ade2-f976bf95df64-kube-api-access-g6jb5\") pod \"cert-manager-cainjector-559cdd8c87-tvhqb\" (UID: \"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.518766 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.518735 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" Sep 07 09:03:54.788789 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:54.786349 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb"] Sep 07 09:03:54.793052 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:03:54.792998 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4f0ec17_9ae0_4f7a_ade2_f976bf95df64.slice/crio-33c972b7396957ea7cbdaa2697115d1551672a4b17e9498a920d5adfcfc684a0 WatchSource:0}: Error finding container 33c972b7396957ea7cbdaa2697115d1551672a4b17e9498a920d5adfcfc684a0: Status 404 returned error can't find the container with id 33c972b7396957ea7cbdaa2697115d1551672a4b17e9498a920d5adfcfc684a0 Sep 07 09:03:55.183809 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:55.183716 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" event={"ID":"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64","Type":"ContainerStarted","Data":"33c972b7396957ea7cbdaa2697115d1551672a4b17e9498a920d5adfcfc684a0"} Sep 07 09:03:57.196851 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:57.196808 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" event={"ID":"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc","Type":"ContainerStarted","Data":"d6b187cf20782ee7eab5cc8653b024aa0af6a3d8527bc7d058b170b782cce875"} Sep 07 09:03:58.202837 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:58.202802 2558 generic.go:356] "Generic (PLEG): container finished" podID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerID="d6b187cf20782ee7eab5cc8653b024aa0af6a3d8527bc7d058b170b782cce875" exitCode=0 Sep 07 09:03:58.203238 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:58.202890 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" event={"ID":"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc","Type":"ContainerDied","Data":"d6b187cf20782ee7eab5cc8653b024aa0af6a3d8527bc7d058b170b782cce875"} Sep 07 09:03:58.204563 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:58.204544 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" event={"ID":"d4f0ec17-9ae0-4f7a-ade2-f976bf95df64","Type":"ContainerStarted","Data":"e72b12c4e345da335cdf6ed151f7b0a6d1eb4199e7a600b043d487106fc254db"} Sep 07 09:03:58.232886 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:58.232847 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-tvhqb" podStartSLOduration=1.658769236 podStartE2EDuration="4.232830634s" totalImagesPullingTime="2.574061398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:03:58.232339864 +0000 UTC m=+296.749130767" watchObservedRunningTime="2026-09-07 09:03:58.232830634 +0000 UTC m=+296.749621537" Sep 07 09:03:59.211233 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:59.211194 2558 generic.go:356] "Generic (PLEG): container finished" podID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerID="7d16f38f6e7fecea75e98f4decf6a256cd67be903efdf79d762564939301f174" exitCode=0 Sep 07 09:03:59.211604 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:03:59.211275 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" event={"ID":"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc","Type":"ContainerDied","Data":"7d16f38f6e7fecea75e98f4decf6a256cd67be903efdf79d762564939301f174"} Sep 07 09:04:00.348418 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.348396 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:04:00.491805 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.491737 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpstj\" (UniqueName: \"kubernetes.io/projected/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-kube-api-access-gpstj\") pod \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " Sep 07 09:04:00.491805 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.491801 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-util\") pod \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " Sep 07 09:04:00.491980 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.491823 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-bundle\") pod \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\" (UID: \"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc\") " Sep 07 09:04:00.492177 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.492144 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-bundle" (OuterVolumeSpecName: "bundle") pod "b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" (UID: "b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:04:00.493780 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.493755 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-kube-api-access-gpstj" (OuterVolumeSpecName: "kube-api-access-gpstj") pod "b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" (UID: "b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc"). InnerVolumeSpecName "kube-api-access-gpstj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:04:00.496505 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.496469 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-util" (OuterVolumeSpecName: "util") pod "b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" (UID: "b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:04:00.593313 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.593262 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpstj\" (UniqueName: \"kubernetes.io/projected/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-kube-api-access-gpstj\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:00.593313 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.593288 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:00.593313 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:00.593320 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:01.225591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.225555 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" event={"ID":"b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc","Type":"ContainerDied","Data":"c42e2340f89975e98265a65b060da98710e47f6d4bd3ffb412834bbf0cd3dac2"} Sep 07 09:04:01.225591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.225581 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv5899" Sep 07 09:04:01.225838 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.225622 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c42e2340f89975e98265a65b060da98710e47f6d4bd3ffb412834bbf0cd3dac2" Sep 07 09:04:01.778719 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.778682 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lzxnr"] Sep 07 09:04:01.779082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.778946 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerName="util" Sep 07 09:04:01.779082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.778957 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerName="util" Sep 07 09:04:01.779082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.778969 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerName="extract" Sep 07 09:04:01.779082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.778974 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerName="extract" Sep 07 09:04:01.779082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.778986 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerName="pull" Sep 07 09:04:01.779082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.778992 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerName="pull" Sep 07 09:04:01.779082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.779042 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0ccd0cf-ba50-4676-8d0d-740dd03e2dbc" containerName="extract" Sep 07 09:04:01.782533 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.782516 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:01.785100 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.785077 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-c9l88\"" Sep 07 09:04:01.789389 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.789366 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lzxnr"] Sep 07 09:04:01.802777 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.802755 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89d18947-f656-4b34-8f39-b0f129649164-bound-sa-token\") pod \"cert-manager-55b4d954d5-lzxnr\" (UID: \"89d18947-f656-4b34-8f39-b0f129649164\") " pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:01.802884 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.802806 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xxlb\" (UniqueName: \"kubernetes.io/projected/89d18947-f656-4b34-8f39-b0f129649164-kube-api-access-9xxlb\") pod \"cert-manager-55b4d954d5-lzxnr\" (UID: \"89d18947-f656-4b34-8f39-b0f129649164\") " pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:01.857451 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.857422 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:04:01.858569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.858547 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:04:01.863285 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.863270 2558 kubelet.go:1648] "Image garbage collection succeeded" Sep 07 09:04:01.903839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.903809 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xxlb\" (UniqueName: \"kubernetes.io/projected/89d18947-f656-4b34-8f39-b0f129649164-kube-api-access-9xxlb\") pod \"cert-manager-55b4d954d5-lzxnr\" (UID: \"89d18947-f656-4b34-8f39-b0f129649164\") " pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:01.903977 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.903873 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89d18947-f656-4b34-8f39-b0f129649164-bound-sa-token\") pod \"cert-manager-55b4d954d5-lzxnr\" (UID: \"89d18947-f656-4b34-8f39-b0f129649164\") " pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:01.911515 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.911486 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89d18947-f656-4b34-8f39-b0f129649164-bound-sa-token\") pod \"cert-manager-55b4d954d5-lzxnr\" (UID: \"89d18947-f656-4b34-8f39-b0f129649164\") " pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:01.911617 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:01.911602 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xxlb\" (UniqueName: \"kubernetes.io/projected/89d18947-f656-4b34-8f39-b0f129649164-kube-api-access-9xxlb\") pod \"cert-manager-55b4d954d5-lzxnr\" (UID: \"89d18947-f656-4b34-8f39-b0f129649164\") " pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:02.094163 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:02.094088 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-c9l88\"" Sep 07 09:04:02.102208 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:02.102183 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lzxnr" Sep 07 09:04:02.368569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:02.368484 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lzxnr"] Sep 07 09:04:02.371257 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:02.371233 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89d18947_f656_4b34_8f39_b0f129649164.slice/crio-bdb0a2d7667af446c7fd563cdaf067a019ec6e41b09311851b30b41196823989 WatchSource:0}: Error finding container bdb0a2d7667af446c7fd563cdaf067a019ec6e41b09311851b30b41196823989: Status 404 returned error can't find the container with id bdb0a2d7667af446c7fd563cdaf067a019ec6e41b09311851b30b41196823989 Sep 07 09:04:03.235219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:03.235178 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lzxnr" event={"ID":"89d18947-f656-4b34-8f39-b0f129649164","Type":"ContainerStarted","Data":"430eade3123123111eda2e1f3b419faa3df042615617664008a71f566927a059"} Sep 07 09:04:03.235219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:03.235225 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lzxnr" event={"ID":"89d18947-f656-4b34-8f39-b0f129649164","Type":"ContainerStarted","Data":"bdb0a2d7667af446c7fd563cdaf067a019ec6e41b09311851b30b41196823989"} Sep 07 09:04:03.251356 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:03.251285 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lzxnr" podStartSLOduration=2.251268199 podStartE2EDuration="2.251268199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:03.24997544 +0000 UTC m=+301.766766343" watchObservedRunningTime="2026-09-07 09:04:03.251268199 +0000 UTC m=+301.768059100" Sep 07 09:04:05.246772 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.245938 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" event={"ID":"fbad3ae7-36da-49af-b5f5-964c4ef47ad6","Type":"ContainerStarted","Data":"02f1831ddcf5b261ec89e14594089907ab40e7b988179c9ea9e24debc93289cf"} Sep 07 09:04:05.246772 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.246733 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:04:05.263234 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.263187 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" podStartSLOduration=1.422191109 podStartE2EDuration="13.263173013s" totalImagesPullingTime="11.840981904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:05.261828682 +0000 UTC m=+303.778619584" watchObservedRunningTime="2026-09-07 09:04:05.263173013 +0000 UTC m=+303.779963916" Sep 07 09:04:05.445071 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.444989 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x"] Sep 07 09:04:05.448289 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.448271 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:05.450651 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.450624 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 07 09:04:05.450777 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.450756 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 07 09:04:05.450863 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.450848 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kn6sm\"" Sep 07 09:04:05.457644 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.457621 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x"] Sep 07 09:04:05.529031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.529002 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77ac6199-037f-43fd-aa42-8e7fdb622b38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8lw4x\" (UID: \"77ac6199-037f-43fd-aa42-8e7fdb622b38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:05.529187 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.529045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvkzm\" (UniqueName: \"kubernetes.io/projected/77ac6199-037f-43fd-aa42-8e7fdb622b38-kube-api-access-gvkzm\") pod \"openshift-lws-operator-fd8ccff4c-8lw4x\" (UID: \"77ac6199-037f-43fd-aa42-8e7fdb622b38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:05.630026 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.629989 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77ac6199-037f-43fd-aa42-8e7fdb622b38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8lw4x\" (UID: \"77ac6199-037f-43fd-aa42-8e7fdb622b38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:05.630186 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.630045 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvkzm\" (UniqueName: \"kubernetes.io/projected/77ac6199-037f-43fd-aa42-8e7fdb622b38-kube-api-access-gvkzm\") pod \"openshift-lws-operator-fd8ccff4c-8lw4x\" (UID: \"77ac6199-037f-43fd-aa42-8e7fdb622b38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:05.630407 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.630386 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77ac6199-037f-43fd-aa42-8e7fdb622b38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8lw4x\" (UID: \"77ac6199-037f-43fd-aa42-8e7fdb622b38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:05.638238 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.638209 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvkzm\" (UniqueName: \"kubernetes.io/projected/77ac6199-037f-43fd-aa42-8e7fdb622b38-kube-api-access-gvkzm\") pod \"openshift-lws-operator-fd8ccff4c-8lw4x\" (UID: \"77ac6199-037f-43fd-aa42-8e7fdb622b38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:05.757974 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:05.757897 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" Sep 07 09:04:06.030960 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:06.030835 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x"] Sep 07 09:04:06.033661 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:06.033630 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77ac6199_037f_43fd_aa42_8e7fdb622b38.slice/crio-226455743f906b64a8c6e14727853c8b90dfd193e141cb1890dc063c30afce7f WatchSource:0}: Error finding container 226455743f906b64a8c6e14727853c8b90dfd193e141cb1890dc063c30afce7f: Status 404 returned error can't find the container with id 226455743f906b64a8c6e14727853c8b90dfd193e141cb1890dc063c30afce7f Sep 07 09:04:06.035618 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:06.035596 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 09:04:06.251720 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:06.251680 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" event={"ID":"77ac6199-037f-43fd-aa42-8e7fdb622b38","Type":"ContainerStarted","Data":"226455743f906b64a8c6e14727853c8b90dfd193e141cb1890dc063c30afce7f"} Sep 07 09:04:09.266277 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:09.266241 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" event={"ID":"77ac6199-037f-43fd-aa42-8e7fdb622b38","Type":"ContainerStarted","Data":"251b545e8a4dd596a342ec1c66e13f0f6a5cbcd7e69126398d42cc99e1b45a23"} Sep 07 09:04:09.282310 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:09.282252 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8lw4x" podStartSLOduration=1.998153236 podStartE2EDuration="4.282235655s" totalImagesPullingTime="2.284082419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:09.281235992 +0000 UTC m=+307.798026896" watchObservedRunningTime="2026-09-07 09:04:09.282235655 +0000 UTC m=+307.799026558" Sep 07 09:04:11.597036 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.597001 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z"] Sep 07 09:04:11.601357 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.601338 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.603743 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.603721 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 07 09:04:11.603743 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.603737 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 07 09:04:11.604624 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.604608 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f5tkk\"" Sep 07 09:04:11.608647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.608252 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z"] Sep 07 09:04:11.679851 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.679819 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.679851 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.679851 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.680028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.679873 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5gmg\" (UniqueName: \"kubernetes.io/projected/223d9e29-908b-495e-9a42-9aa395bc2d00-kube-api-access-l5gmg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.780945 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.780917 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.780945 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.780949 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.781144 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.780972 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5gmg\" (UniqueName: \"kubernetes.io/projected/223d9e29-908b-495e-9a42-9aa395bc2d00-kube-api-access-l5gmg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.781407 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.781385 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.781449 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.781394 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.788842 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.788818 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5gmg\" (UniqueName: \"kubernetes.io/projected/223d9e29-908b-495e-9a42-9aa395bc2d00-kube-api-access-l5gmg\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:11.911528 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:11.911447 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:12.186471 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:12.186445 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z"] Sep 07 09:04:12.188643 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:12.188617 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223d9e29_908b_495e_9a42_9aa395bc2d00.slice/crio-5757733dfca5e2dcf613874dbd764b71985c1a06a788e6f84fefe73e6df58822 WatchSource:0}: Error finding container 5757733dfca5e2dcf613874dbd764b71985c1a06a788e6f84fefe73e6df58822: Status 404 returned error can't find the container with id 5757733dfca5e2dcf613874dbd764b71985c1a06a788e6f84fefe73e6df58822 Sep 07 09:04:12.257743 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:12.257719 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9lxh7" Sep 07 09:04:12.282588 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:12.282554 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" event={"ID":"223d9e29-908b-495e-9a42-9aa395bc2d00","Type":"ContainerStarted","Data":"597ba6dc0992ddd8fb8e641274c5600b35490008a79748411ce08eac9d78bacd"} Sep 07 09:04:12.282742 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:12.282607 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" event={"ID":"223d9e29-908b-495e-9a42-9aa395bc2d00","Type":"ContainerStarted","Data":"5757733dfca5e2dcf613874dbd764b71985c1a06a788e6f84fefe73e6df58822"} Sep 07 09:04:13.287544 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:13.287510 2558 generic.go:356] "Generic (PLEG): container finished" podID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerID="597ba6dc0992ddd8fb8e641274c5600b35490008a79748411ce08eac9d78bacd" exitCode=0 Sep 07 09:04:13.287949 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:13.287589 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" event={"ID":"223d9e29-908b-495e-9a42-9aa395bc2d00","Type":"ContainerDied","Data":"597ba6dc0992ddd8fb8e641274c5600b35490008a79748411ce08eac9d78bacd"} Sep 07 09:04:14.299281 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:14.299240 2558 generic.go:356] "Generic (PLEG): container finished" podID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerID="ccb6135e45bbb5a4276458d894564d55e44320d0ed15bb3b05ac9c1d4bd0d617" exitCode=0 Sep 07 09:04:14.299754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:14.299358 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" event={"ID":"223d9e29-908b-495e-9a42-9aa395bc2d00","Type":"ContainerDied","Data":"ccb6135e45bbb5a4276458d894564d55e44320d0ed15bb3b05ac9c1d4bd0d617"} Sep 07 09:04:15.306194 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:15.306158 2558 generic.go:356] "Generic (PLEG): container finished" podID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerID="8487471b576c8a646cd693837e6b3dd4044d4f0b5f26abe31a266bc2e6f743b5" exitCode=0 Sep 07 09:04:15.306599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:15.306243 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" event={"ID":"223d9e29-908b-495e-9a42-9aa395bc2d00","Type":"ContainerDied","Data":"8487471b576c8a646cd693837e6b3dd4044d4f0b5f26abe31a266bc2e6f743b5"} Sep 07 09:04:16.447580 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.447553 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:16.517965 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.517936 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-util\") pod \"223d9e29-908b-495e-9a42-9aa395bc2d00\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " Sep 07 09:04:16.517965 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.517968 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5gmg\" (UniqueName: \"kubernetes.io/projected/223d9e29-908b-495e-9a42-9aa395bc2d00-kube-api-access-l5gmg\") pod \"223d9e29-908b-495e-9a42-9aa395bc2d00\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " Sep 07 09:04:16.518140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.518031 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-bundle\") pod \"223d9e29-908b-495e-9a42-9aa395bc2d00\" (UID: \"223d9e29-908b-495e-9a42-9aa395bc2d00\") " Sep 07 09:04:16.518879 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.518855 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-bundle" (OuterVolumeSpecName: "bundle") pod "223d9e29-908b-495e-9a42-9aa395bc2d00" (UID: "223d9e29-908b-495e-9a42-9aa395bc2d00"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:04:16.520065 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.520039 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223d9e29-908b-495e-9a42-9aa395bc2d00-kube-api-access-l5gmg" (OuterVolumeSpecName: "kube-api-access-l5gmg") pod "223d9e29-908b-495e-9a42-9aa395bc2d00" (UID: "223d9e29-908b-495e-9a42-9aa395bc2d00"). InnerVolumeSpecName "kube-api-access-l5gmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:04:16.523937 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.523916 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-util" (OuterVolumeSpecName: "util") pod "223d9e29-908b-495e-9a42-9aa395bc2d00" (UID: "223d9e29-908b-495e-9a42-9aa395bc2d00"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:04:16.619207 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.619134 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:16.619207 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.619159 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5gmg\" (UniqueName: \"kubernetes.io/projected/223d9e29-908b-495e-9a42-9aa395bc2d00-kube-api-access-l5gmg\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:16.619207 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:16.619169 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/223d9e29-908b-495e-9a42-9aa395bc2d00-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:17.317234 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:17.317197 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" event={"ID":"223d9e29-908b-495e-9a42-9aa395bc2d00","Type":"ContainerDied","Data":"5757733dfca5e2dcf613874dbd764b71985c1a06a788e6f84fefe73e6df58822"} Sep 07 09:04:17.317414 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:17.317246 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afvr66z" Sep 07 09:04:17.317414 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:17.317251 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5757733dfca5e2dcf613874dbd764b71985c1a06a788e6f84fefe73e6df58822" Sep 07 09:04:28.343701 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.343668 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h"] Sep 07 09:04:28.344213 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.344014 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerName="extract" Sep 07 09:04:28.344213 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.344029 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerName="extract" Sep 07 09:04:28.344213 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.344042 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerName="util" Sep 07 09:04:28.344213 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.344048 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerName="util" Sep 07 09:04:28.344213 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.344060 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerName="pull" Sep 07 09:04:28.344213 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.344065 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerName="pull" Sep 07 09:04:28.344213 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.344120 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="223d9e29-908b-495e-9a42-9aa395bc2d00" containerName="extract" Sep 07 09:04:28.396917 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.396885 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h"] Sep 07 09:04:28.397067 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.396981 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.400430 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.400399 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 07 09:04:28.400632 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.400610 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c5k27\"" Sep 07 09:04:28.400741 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.400634 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 07 09:04:28.400741 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.400662 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 07 09:04:28.400741 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.400634 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 07 09:04:28.507244 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.507213 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18632cb3-b012-4464-bcc6-73b39d9f6227-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.507419 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.507261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18632cb3-b012-4464-bcc6-73b39d9f6227-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.507419 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.507371 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18632cb3-b012-4464-bcc6-73b39d9f6227-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.507419 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.507406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18632cb3-b012-4464-bcc6-73b39d9f6227-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.507546 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.507428 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t84r\" (UniqueName: \"kubernetes.io/projected/18632cb3-b012-4464-bcc6-73b39d9f6227-kube-api-access-6t84r\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.608165 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.608066 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t84r\" (UniqueName: \"kubernetes.io/projected/18632cb3-b012-4464-bcc6-73b39d9f6227-kube-api-access-6t84r\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.608165 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.608132 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18632cb3-b012-4464-bcc6-73b39d9f6227-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.608165 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.608161 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18632cb3-b012-4464-bcc6-73b39d9f6227-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.608554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.608207 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18632cb3-b012-4464-bcc6-73b39d9f6227-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.608554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.608232 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18632cb3-b012-4464-bcc6-73b39d9f6227-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.608762 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.608730 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/18632cb3-b012-4464-bcc6-73b39d9f6227-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.610653 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.610631 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18632cb3-b012-4464-bcc6-73b39d9f6227-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.610887 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.610865 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/18632cb3-b012-4464-bcc6-73b39d9f6227-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.610930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.610870 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/18632cb3-b012-4464-bcc6-73b39d9f6227-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.627486 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.627461 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj"] Sep 07 09:04:28.627963 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.627945 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t84r\" (UniqueName: \"kubernetes.io/projected/18632cb3-b012-4464-bcc6-73b39d9f6227-kube-api-access-6t84r\") pod \"opendatahub-operator-controller-manager-78c59b5944-wpx8h\" (UID: \"18632cb3-b012-4464-bcc6-73b39d9f6227\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.644056 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.644034 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj"] Sep 07 09:04:28.644157 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.644133 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.646705 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.646667 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 07 09:04:28.646820 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.646678 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 07 09:04:28.646820 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.646759 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 07 09:04:28.646928 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.646846 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c8bwd\"" Sep 07 09:04:28.707781 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.707749 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:28.709086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.709063 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff187995-eab2-4c9f-beef-a9e53c21a1bc-cert\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.709174 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.709118 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff187995-eab2-4c9f-beef-a9e53c21a1bc-metrics-cert\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.709174 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.709158 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5fz9\" (UniqueName: \"kubernetes.io/projected/ff187995-eab2-4c9f-beef-a9e53c21a1bc-kube-api-access-d5fz9\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.709278 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.709188 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ff187995-eab2-4c9f-beef-a9e53c21a1bc-manager-config\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.810045 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.810013 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff187995-eab2-4c9f-beef-a9e53c21a1bc-cert\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.810203 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.810081 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff187995-eab2-4c9f-beef-a9e53c21a1bc-metrics-cert\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.810203 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.810111 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5fz9\" (UniqueName: \"kubernetes.io/projected/ff187995-eab2-4c9f-beef-a9e53c21a1bc-kube-api-access-d5fz9\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.810203 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.810142 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ff187995-eab2-4c9f-beef-a9e53c21a1bc-manager-config\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.810807 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.810783 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ff187995-eab2-4c9f-beef-a9e53c21a1bc-manager-config\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.812403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.812385 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff187995-eab2-4c9f-beef-a9e53c21a1bc-metrics-cert\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.812403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.812398 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff187995-eab2-4c9f-beef-a9e53c21a1bc-cert\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.817917 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.817893 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5fz9\" (UniqueName: \"kubernetes.io/projected/ff187995-eab2-4c9f-beef-a9e53c21a1bc-kube-api-access-d5fz9\") pod \"lws-controller-manager-94f7b5c54-xnldj\" (UID: \"ff187995-eab2-4c9f-beef-a9e53c21a1bc\") " pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:28.953320 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:28.953206 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:29.000048 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.000025 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h"] Sep 07 09:04:29.003093 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:29.003068 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18632cb3_b012_4464_bcc6_73b39d9f6227.slice/crio-fc673c3277c97f934d416e5e19975d30d344f4d8192d4c7426f06baea3063582 WatchSource:0}: Error finding container fc673c3277c97f934d416e5e19975d30d344f4d8192d4c7426f06baea3063582: Status 404 returned error can't find the container with id fc673c3277c97f934d416e5e19975d30d344f4d8192d4c7426f06baea3063582 Sep 07 09:04:29.013917 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.013896 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5"] Sep 07 09:04:29.041489 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.041439 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5"] Sep 07 09:04:29.041614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.041581 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.045392 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.044965 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 07 09:04:29.045392 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.045237 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 07 09:04:29.045995 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.045739 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f5tkk\"" Sep 07 09:04:29.112590 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.112560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jktmd\" (UniqueName: \"kubernetes.io/projected/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-kube-api-access-jktmd\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.112779 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.112608 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.112779 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.112702 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.213325 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.213237 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.213325 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.213283 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jktmd\" (UniqueName: \"kubernetes.io/projected/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-kube-api-access-jktmd\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.213528 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.213338 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.213612 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.213595 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.213654 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.213632 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.220592 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.220568 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jktmd\" (UniqueName: \"kubernetes.io/projected/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-kube-api-access-jktmd\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.259243 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.259215 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj"] Sep 07 09:04:29.261449 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:29.261426 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff187995_eab2_4c9f_beef_a9e53c21a1bc.slice/crio-d80bdf32b2dadb684bc82867281a739327a9528729198c89855bd15464e72948 WatchSource:0}: Error finding container d80bdf32b2dadb684bc82867281a739327a9528729198c89855bd15464e72948: Status 404 returned error can't find the container with id d80bdf32b2dadb684bc82867281a739327a9528729198c89855bd15464e72948 Sep 07 09:04:29.354542 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.354507 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:29.374861 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.374832 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" event={"ID":"18632cb3-b012-4464-bcc6-73b39d9f6227","Type":"ContainerStarted","Data":"fc673c3277c97f934d416e5e19975d30d344f4d8192d4c7426f06baea3063582"} Sep 07 09:04:29.376125 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.376096 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" event={"ID":"ff187995-eab2-4c9f-beef-a9e53c21a1bc","Type":"ContainerStarted","Data":"d80bdf32b2dadb684bc82867281a739327a9528729198c89855bd15464e72948"} Sep 07 09:04:29.628630 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:29.628526 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5"] Sep 07 09:04:29.631893 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:29.631863 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdfa6667_a34d_4c6f_bee4_672c7b3e41b8.slice/crio-dab6df594ed0e47ba0220e047c3a6d76e78aaf78718f29a274f47eebdf55b6a1 WatchSource:0}: Error finding container dab6df594ed0e47ba0220e047c3a6d76e78aaf78718f29a274f47eebdf55b6a1: Status 404 returned error can't find the container with id dab6df594ed0e47ba0220e047c3a6d76e78aaf78718f29a274f47eebdf55b6a1 Sep 07 09:04:30.385841 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:30.385796 2558 generic.go:356] "Generic (PLEG): container finished" podID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerID="2cff1287628d8d541ea0f923822b163c0d4d2b1bc50a98a35248ac4820ea5dbb" exitCode=0 Sep 07 09:04:30.386355 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:30.385913 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" event={"ID":"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8","Type":"ContainerDied","Data":"2cff1287628d8d541ea0f923822b163c0d4d2b1bc50a98a35248ac4820ea5dbb"} Sep 07 09:04:30.386355 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:30.385964 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" event={"ID":"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8","Type":"ContainerStarted","Data":"dab6df594ed0e47ba0220e047c3a6d76e78aaf78718f29a274f47eebdf55b6a1"} Sep 07 09:04:33.402181 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:33.402144 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" event={"ID":"ff187995-eab2-4c9f-beef-a9e53c21a1bc","Type":"ContainerStarted","Data":"5535e35eeccef9630e27bbebefc35f492a7dc70c1519fd0cfd5869e3a2034cb9"} Sep 07 09:04:33.402593 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:33.402199 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:33.403905 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:33.403878 2558 generic.go:356] "Generic (PLEG): container finished" podID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerID="6b46744ec0b5abd4fd70c5d4417ad0b86ffc13c60aaedb304d15ba61f4a64aca" exitCode=0 Sep 07 09:04:33.403980 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:33.403965 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" event={"ID":"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8","Type":"ContainerDied","Data":"6b46744ec0b5abd4fd70c5d4417ad0b86ffc13c60aaedb304d15ba61f4a64aca"} Sep 07 09:04:33.405706 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:33.405682 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" event={"ID":"18632cb3-b012-4464-bcc6-73b39d9f6227","Type":"ContainerStarted","Data":"21d0b5ac40303f453ad349d047afe485c5cae0c39f3608a169ef2b5c82417dec"} Sep 07 09:04:33.419238 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:33.419200 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" podStartSLOduration=2.298092447 podStartE2EDuration="5.419185814s" totalImagesPullingTime="3.121093367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:33.417232431 +0000 UTC m=+331.934023335" watchObservedRunningTime="2026-09-07 09:04:33.419185814 +0000 UTC m=+331.935976717" Sep 07 09:04:34.412502 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:34.412468 2558 generic.go:356] "Generic (PLEG): container finished" podID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerID="019a9d9cfbe2fd2f47bc4333623e4c0c6db71cac4d40db7664d0a2b363294db0" exitCode=0 Sep 07 09:04:34.412937 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:34.412547 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" event={"ID":"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8","Type":"ContainerDied","Data":"019a9d9cfbe2fd2f47bc4333623e4c0c6db71cac4d40db7664d0a2b363294db0"} Sep 07 09:04:34.414057 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:34.414033 2558 generic.go:356] "Generic (PLEG): container finished" podID="18632cb3-b012-4464-bcc6-73b39d9f6227" containerID="21d0b5ac40303f453ad349d047afe485c5cae0c39f3608a169ef2b5c82417dec" exitCode=0 Sep 07 09:04:34.414152 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:34.414118 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" event={"ID":"18632cb3-b012-4464-bcc6-73b39d9f6227","Type":"ContainerDied","Data":"21d0b5ac40303f453ad349d047afe485c5cae0c39f3608a169ef2b5c82417dec"} Sep 07 09:04:35.420263 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.420222 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" event={"ID":"18632cb3-b012-4464-bcc6-73b39d9f6227","Type":"ContainerStarted","Data":"72e3410bea9119fb1afa7295b55c4a3e4349e8af0abda2a8e9346ab75092655c"} Sep 07 09:04:35.420666 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.420473 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:35.440205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.440151 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" podStartSLOduration=2.579427912 podStartE2EDuration="7.440135244s" totalImagesPullingTime="3.710710714s" totalInitContainerRuntime="1.149996618s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:35.438953845 +0000 UTC m=+333.955744771" watchObservedRunningTime="2026-09-07 09:04:35.440135244 +0000 UTC m=+333.956926157" Sep 07 09:04:35.549762 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.549732 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:35.675519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.675431 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-util\") pod \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " Sep 07 09:04:35.675519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.675480 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jktmd\" (UniqueName: \"kubernetes.io/projected/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-kube-api-access-jktmd\") pod \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " Sep 07 09:04:35.675519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.675508 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-bundle\") pod \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\" (UID: \"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8\") " Sep 07 09:04:35.676326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.676273 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-bundle" (OuterVolumeSpecName: "bundle") pod "fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" (UID: "fdfa6667-a34d-4c6f-bee4-672c7b3e41b8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:04:35.677665 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.677637 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-kube-api-access-jktmd" (OuterVolumeSpecName: "kube-api-access-jktmd") pod "fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" (UID: "fdfa6667-a34d-4c6f-bee4-672c7b3e41b8"). InnerVolumeSpecName "kube-api-access-jktmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:04:35.680097 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.680063 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-util" (OuterVolumeSpecName: "util") pod "fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" (UID: "fdfa6667-a34d-4c6f-bee4-672c7b3e41b8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:04:35.776315 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.776256 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:35.776315 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.776293 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jktmd\" (UniqueName: \"kubernetes.io/projected/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-kube-api-access-jktmd\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:35.776315 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:35.776325 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fdfa6667-a34d-4c6f-bee4-672c7b3e41b8-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:04:36.426469 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:36.426426 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" event={"ID":"fdfa6667-a34d-4c6f-bee4-672c7b3e41b8","Type":"ContainerDied","Data":"dab6df594ed0e47ba0220e047c3a6d76e78aaf78718f29a274f47eebdf55b6a1"} Sep 07 09:04:36.426823 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:36.426488 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dab6df594ed0e47ba0220e047c3a6d76e78aaf78718f29a274f47eebdf55b6a1" Sep 07 09:04:36.426823 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:36.426446 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06z4l5" Sep 07 09:04:44.416851 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:44.416822 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-94f7b5c54-xnldj" Sep 07 09:04:46.428941 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:46.428908 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-wpx8h" Sep 07 09:04:55.054878 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.054790 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2"] Sep 07 09:04:55.055264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.055109 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerName="extract" Sep 07 09:04:55.055264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.055122 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerName="extract" Sep 07 09:04:55.055264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.055136 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerName="pull" Sep 07 09:04:55.055264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.055141 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerName="pull" Sep 07 09:04:55.055264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.055151 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerName="util" Sep 07 09:04:55.055264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.055156 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerName="util" Sep 07 09:04:55.055264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.055220 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdfa6667-a34d-4c6f-bee4-672c7b3e41b8" containerName="extract" Sep 07 09:04:55.057941 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.057924 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.060587 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.060565 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7x28j\"" Sep 07 09:04:55.060698 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.060643 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 07 09:04:55.060780 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.060708 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 07 09:04:55.060780 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.060711 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 07 09:04:55.070813 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.070794 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2"] Sep 07 09:04:55.237218 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237185 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237227 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237253 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237354 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237386 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237413 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237589 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237480 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237589 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237506 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.237589 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.237554 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brchv\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-kube-api-access-brchv\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338469 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338382 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338469 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338425 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338469 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338467 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338492 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338518 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brchv\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-kube-api-access-brchv\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338556 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338592 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338618 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.338759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338667 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.339008 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.338837 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.339130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.339108 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.339173 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.339133 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.339219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.339176 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.339431 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.339411 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.341069 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.341046 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.341069 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.341060 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.348829 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.348810 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.349021 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.349004 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brchv\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-kube-api-access-brchv\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.367969 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.367948 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:04:55.373672 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.373650 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6"] Sep 07 09:04:55.376936 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.376916 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.391646 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.391620 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6"] Sep 07 09:04:55.540944 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.540907 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.540944 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.540947 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.541169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.540974 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckh7h\" (UniqueName: \"kubernetes.io/projected/ec08351a-7ba8-48b2-b010-a946abe2fa52-kube-api-access-ckh7h\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.541169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.541026 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.541169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.541045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.541169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.541074 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.541169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.541143 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec08351a-7ba8-48b2-b010-a946abe2fa52-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.541365 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.541183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.541365 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.541200 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.641883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.641804 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.641883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.641847 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec08351a-7ba8-48b2-b010-a946abe2fa52-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.641883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.641879 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.641912 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.641958 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.641986 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642026 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckh7h\" (UniqueName: \"kubernetes.io/projected/ec08351a-7ba8-48b2-b010-a946abe2fa52-kube-api-access-ckh7h\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642062 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642414 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642349 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642649 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642622 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642783 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642719 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec08351a-7ba8-48b2-b010-a946abe2fa52-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642783 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642715 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.642783 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.642756 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.644309 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.644275 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.644438 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.644422 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.654653 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.654629 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec08351a-7ba8-48b2-b010-a946abe2fa52-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.654811 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.654792 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckh7h\" (UniqueName: \"kubernetes.io/projected/ec08351a-7ba8-48b2-b010-a946abe2fa52-kube-api-access-ckh7h\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6\" (UID: \"ec08351a-7ba8-48b2-b010-a946abe2fa52\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.663244 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.663224 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2"] Sep 07 09:04:55.665891 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:55.665867 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b57c723_7f69_46fb_974b_a10f13d5d3e9.slice/crio-7320ffb65d1f75235c8dc21de41f09f086be61d0e6b701028330a3d6fa7a8ace WatchSource:0}: Error finding container 7320ffb65d1f75235c8dc21de41f09f086be61d0e6b701028330a3d6fa7a8ace: Status 404 returned error can't find the container with id 7320ffb65d1f75235c8dc21de41f09f086be61d0e6b701028330a3d6fa7a8ace Sep 07 09:04:55.701575 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.701549 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:55.975250 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:55.975226 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6"] Sep 07 09:04:55.977594 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:04:55.977568 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec08351a_7ba8_48b2_b010_a946abe2fa52.slice/crio-3c2fc86d224b66cca422618615a3b56c441c0a54b108c4d033a55346eff71003 WatchSource:0}: Error finding container 3c2fc86d224b66cca422618615a3b56c441c0a54b108c4d033a55346eff71003: Status 404 returned error can't find the container with id 3c2fc86d224b66cca422618615a3b56c441c0a54b108c4d033a55346eff71003 Sep 07 09:04:56.526837 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:56.526797 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" event={"ID":"ec08351a-7ba8-48b2-b010-a946abe2fa52","Type":"ContainerStarted","Data":"3c2fc86d224b66cca422618615a3b56c441c0a54b108c4d033a55346eff71003"} Sep 07 09:04:56.527964 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:56.527938 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" event={"ID":"5b57c723-7f69-46fb-974b-a10f13d5d3e9","Type":"ContainerStarted","Data":"7320ffb65d1f75235c8dc21de41f09f086be61d0e6b701028330a3d6fa7a8ace"} Sep 07 09:04:58.761326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:58.761268 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:04:58.761617 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:58.761370 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:04:58.761617 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:58.761430 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:04:58.767874 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:58.767839 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:04:58.767928 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:58.767906 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:04:58.767969 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:58.767936 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:04:59.321926 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.321893 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s5bdn"] Sep 07 09:04:59.325008 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.324991 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:04:59.327629 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.327610 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Sep 07 09:04:59.327755 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.327710 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lkrbl\"" Sep 07 09:04:59.334205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.334144 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s5bdn"] Sep 07 09:04:59.478271 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.478235 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11896531-65c0-4f71-98c3-eedd91d81900-cert\") pod \"odh-model-controller-7998c67dc4-s5bdn\" (UID: \"11896531-65c0-4f71-98c3-eedd91d81900\") " pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:04:59.478465 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.478280 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zhw8\" (UniqueName: \"kubernetes.io/projected/11896531-65c0-4f71-98c3-eedd91d81900-kube-api-access-5zhw8\") pod \"odh-model-controller-7998c67dc4-s5bdn\" (UID: \"11896531-65c0-4f71-98c3-eedd91d81900\") " pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:04:59.545658 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.545619 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" event={"ID":"5b57c723-7f69-46fb-974b-a10f13d5d3e9","Type":"ContainerStarted","Data":"c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07"} Sep 07 09:04:59.547197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.547169 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" event={"ID":"ec08351a-7ba8-48b2-b010-a946abe2fa52","Type":"ContainerStarted","Data":"711276bdf6839fd468cb8fa69877e99096cfc37c406ca56c6f24b67a0a10d1f2"} Sep 07 09:04:59.565915 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.565867 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" podStartSLOduration=1.4661440049999999 podStartE2EDuration="4.565852677s" totalImagesPullingTime="3.099708672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:59.564588845 +0000 UTC m=+358.081379774" watchObservedRunningTime="2026-09-07 09:04:59.565852677 +0000 UTC m=+358.082643580" Sep 07 09:04:59.578857 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.578789 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zhw8\" (UniqueName: \"kubernetes.io/projected/11896531-65c0-4f71-98c3-eedd91d81900-kube-api-access-5zhw8\") pod \"odh-model-controller-7998c67dc4-s5bdn\" (UID: \"11896531-65c0-4f71-98c3-eedd91d81900\") " pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:04:59.578956 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.578910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11896531-65c0-4f71-98c3-eedd91d81900-cert\") pod \"odh-model-controller-7998c67dc4-s5bdn\" (UID: \"11896531-65c0-4f71-98c3-eedd91d81900\") " pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:04:59.579053 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:04:59.579038 2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Sep 07 09:04:59.579113 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:04:59.579102 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11896531-65c0-4f71-98c3-eedd91d81900-cert podName:11896531-65c0-4f71-98c3-eedd91d81900 nodeName:}" failed. No retries permitted until 2026-09-07 09:05:00.079082281 +0000 UTC m=+358.595873180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/11896531-65c0-4f71-98c3-eedd91d81900-cert") pod "odh-model-controller-7998c67dc4-s5bdn" (UID: "11896531-65c0-4f71-98c3-eedd91d81900") : secret "odh-model-controller-webhook-cert" not found Sep 07 09:04:59.585471 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.585431 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" podStartSLOduration=1.803894254 podStartE2EDuration="4.585417402s" totalImagesPullingTime="2.781523148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:59.583690385 +0000 UTC m=+358.100481324" watchObservedRunningTime="2026-09-07 09:04:59.585417402 +0000 UTC m=+358.102208343" Sep 07 09:04:59.589211 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.589188 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zhw8\" (UniqueName: \"kubernetes.io/projected/11896531-65c0-4f71-98c3-eedd91d81900-kube-api-access-5zhw8\") pod \"odh-model-controller-7998c67dc4-s5bdn\" (UID: \"11896531-65c0-4f71-98c3-eedd91d81900\") " pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:04:59.702387 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.702288 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:59.706377 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.706353 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:04:59.708725 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.708703 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lkmxx"] Sep 07 09:04:59.711910 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.711892 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:04:59.713919 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.713899 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Sep 07 09:04:59.722330 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.722310 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lkmxx"] Sep 07 09:04:59.881079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.880997 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6lp8\" (UniqueName: \"kubernetes.io/projected/6f5866f9-d63e-43d6-8cad-1c0e418fd464-kube-api-access-s6lp8\") pod \"model-serving-api-54d89bd5fc-lkmxx\" (UID: \"6f5866f9-d63e-43d6-8cad-1c0e418fd464\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:04:59.881079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.881034 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5866f9-d63e-43d6-8cad-1c0e418fd464-tls-certs\") pod \"model-serving-api-54d89bd5fc-lkmxx\" (UID: \"6f5866f9-d63e-43d6-8cad-1c0e418fd464\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:04:59.982101 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.982068 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6lp8\" (UniqueName: \"kubernetes.io/projected/6f5866f9-d63e-43d6-8cad-1c0e418fd464-kube-api-access-s6lp8\") pod \"model-serving-api-54d89bd5fc-lkmxx\" (UID: \"6f5866f9-d63e-43d6-8cad-1c0e418fd464\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:04:59.982332 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.982117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5866f9-d63e-43d6-8cad-1c0e418fd464-tls-certs\") pod \"model-serving-api-54d89bd5fc-lkmxx\" (UID: \"6f5866f9-d63e-43d6-8cad-1c0e418fd464\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:04:59.984833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.984805 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f5866f9-d63e-43d6-8cad-1c0e418fd464-tls-certs\") pod \"model-serving-api-54d89bd5fc-lkmxx\" (UID: \"6f5866f9-d63e-43d6-8cad-1c0e418fd464\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:04:59.990228 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:04:59.990203 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6lp8\" (UniqueName: \"kubernetes.io/projected/6f5866f9-d63e-43d6-8cad-1c0e418fd464-kube-api-access-s6lp8\") pod \"model-serving-api-54d89bd5fc-lkmxx\" (UID: \"6f5866f9-d63e-43d6-8cad-1c0e418fd464\") " pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:05:00.023072 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.023043 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:05:00.083025 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.082994 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11896531-65c0-4f71-98c3-eedd91d81900-cert\") pod \"odh-model-controller-7998c67dc4-s5bdn\" (UID: \"11896531-65c0-4f71-98c3-eedd91d81900\") " pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:05:00.085769 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.085745 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11896531-65c0-4f71-98c3-eedd91d81900-cert\") pod \"odh-model-controller-7998c67dc4-s5bdn\" (UID: \"11896531-65c0-4f71-98c3-eedd91d81900\") " pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:05:00.237223 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.237142 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:05:00.296490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.296458 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lkmxx"] Sep 07 09:05:00.300713 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:05:00.300686 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5866f9_d63e_43d6_8cad_1c0e418fd464.slice/crio-d1de2c15e59e0f84c19c8c714d46d47b617d711015736734e856f36903fb6a92 WatchSource:0}: Error finding container d1de2c15e59e0f84c19c8c714d46d47b617d711015736734e856f36903fb6a92: Status 404 returned error can't find the container with id d1de2c15e59e0f84c19c8c714d46d47b617d711015736734e856f36903fb6a92 Sep 07 09:05:00.368990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.368956 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:05:00.370229 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.370202 2558 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 07 09:05:00.370368 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.370256 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 07 09:05:00.507331 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.507279 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s5bdn"] Sep 07 09:05:00.509249 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:05:00.509220 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11896531_65c0_4f71_98c3_eedd91d81900.slice/crio-f2713f7b752783f7aecb6bba6a2f9be1cedd7ec636d2f9e6193bba8d95ccadab WatchSource:0}: Error finding container f2713f7b752783f7aecb6bba6a2f9be1cedd7ec636d2f9e6193bba8d95ccadab: Status 404 returned error can't find the container with id f2713f7b752783f7aecb6bba6a2f9be1cedd7ec636d2f9e6193bba8d95ccadab Sep 07 09:05:00.553527 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.553494 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" event={"ID":"11896531-65c0-4f71-98c3-eedd91d81900","Type":"ContainerStarted","Data":"f2713f7b752783f7aecb6bba6a2f9be1cedd7ec636d2f9e6193bba8d95ccadab"} Sep 07 09:05:00.554741 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.554720 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" event={"ID":"6f5866f9-d63e-43d6-8cad-1c0e418fd464","Type":"ContainerStarted","Data":"d1de2c15e59e0f84c19c8c714d46d47b617d711015736734e856f36903fb6a92"} Sep 07 09:05:00.555076 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.555058 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:05:00.555827 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.555810 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6" Sep 07 09:05:00.607075 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:00.605202 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2"] Sep 07 09:05:01.368557 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:01.368513 2558 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 07 09:05:01.368997 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:01.368588 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 07 09:05:02.368797 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:02.368756 2558 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 07 09:05:02.369274 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:02.368825 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 07 09:05:02.568443 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:02.568381 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerName="istio-proxy" containerID="cri-o://c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07" gracePeriod=30 Sep 07 09:05:04.585017 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:04.584962 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" event={"ID":"11896531-65c0-4f71-98c3-eedd91d81900","Type":"ContainerStarted","Data":"139649db7828588e76c67bef8f87816dd729fdfeb53c74b53337bfb16cec4e49"} Sep 07 09:05:04.585539 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:04.585070 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:05:04.586520 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:04.586494 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" event={"ID":"6f5866f9-d63e-43d6-8cad-1c0e418fd464","Type":"ContainerStarted","Data":"3050c95964bf6f67e9bec4876ba974219b0e1b358a0ffdb697cdd2a366c16175"} Sep 07 09:05:04.586613 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:04.586561 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:05:04.600189 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:04.600142 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" podStartSLOduration=1.880716532 podStartE2EDuration="5.600128943s" totalImagesPullingTime="3.719412411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:05:04.599895458 +0000 UTC m=+363.116686361" watchObservedRunningTime="2026-09-07 09:05:04.600128943 +0000 UTC m=+363.116919845" Sep 07 09:05:04.614791 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:04.614736 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" podStartSLOduration=1.733171631 podStartE2EDuration="5.614720449s" totalImagesPullingTime="3.881548818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:05:04.613877008 +0000 UTC m=+363.130667923" watchObservedRunningTime="2026-09-07 09:05:04.614720449 +0000 UTC m=+363.131511352" Sep 07 09:05:05.826113 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.826082 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54"] Sep 07 09:05:05.833582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.833554 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:05.836210 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.836183 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 07 09:05:05.836355 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.836187 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 07 09:05:05.836355 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.836234 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ccv6f\"" Sep 07 09:05:05.836469 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.836373 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 07 09:05:05.843452 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.843430 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54"] Sep 07 09:05:05.934710 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.934674 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hr8l\" (UniqueName: \"kubernetes.io/projected/b24df2d2-d777-47ba-9509-c3b16745fce7-kube-api-access-8hr8l\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:05.934905 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.934724 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:05.934905 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.934817 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:05.934905 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.934855 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:05.934905 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.934897 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24df2d2-d777-47ba-9509-c3b16745fce7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:05.935111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.934929 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:05.935111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:05.934961 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.035540 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.035501 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.035540 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.035539 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hr8l\" (UniqueName: \"kubernetes.io/projected/b24df2d2-d777-47ba-9509-c3b16745fce7-kube-api-access-8hr8l\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.035765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.035561 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.035765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.035619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.035765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.035641 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.035765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.035680 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24df2d2-d777-47ba-9509-c3b16745fce7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.035765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.035713 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.036266 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.036237 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.038095 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.038063 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.038268 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.038246 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.038471 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.038455 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b24df2d2-d777-47ba-9509-c3b16745fce7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.038509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.038492 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.043599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.043573 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hr8l\" (UniqueName: \"kubernetes.io/projected/b24df2d2-d777-47ba-9509-c3b16745fce7-kube-api-access-8hr8l\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.043874 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.043856 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b24df2d2-d777-47ba-9509-c3b16745fce7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-wxd54\" (UID: \"b24df2d2-d777-47ba-9509-c3b16745fce7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.145582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.145506 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:06.504249 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.504208 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54"] Sep 07 09:05:06.509951 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:05:06.509925 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24df2d2_d777_47ba_9509_c3b16745fce7.slice/crio-3e93674954604432e669fc7122b81165b1d3f6360cbfb98eea92725d39e84b85 WatchSource:0}: Error finding container 3e93674954604432e669fc7122b81165b1d3f6360cbfb98eea92725d39e84b85: Status 404 returned error can't find the container with id 3e93674954604432e669fc7122b81165b1d3f6360cbfb98eea92725d39e84b85 Sep 07 09:05:06.597180 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:06.597122 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" event={"ID":"b24df2d2-d777-47ba-9509-c3b16745fce7","Type":"ContainerStarted","Data":"3e93674954604432e669fc7122b81165b1d3f6360cbfb98eea92725d39e84b85"} Sep 07 09:05:07.844317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.844243 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:05:07.950587 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950557 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-socket\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950770 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950602 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-token\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950770 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950644 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-credential-socket\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950770 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950705 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istiod-ca-cert\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950770 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950736 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brchv\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-kube-api-access-brchv\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950792 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-certs\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950853 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-envoy\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950895 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-podinfo\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.950996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950876 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:05:07.950996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950935 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-data\") pod \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\" (UID: \"5b57c723-7f69-46fb-974b-a10f13d5d3e9\") " Sep 07 09:05:07.951239 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.950962 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:05:07.951239 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.951117 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:05:07.951239 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.951212 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:05:07.951408 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.951353 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:07.951408 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.951372 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:07.951408 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.951385 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:07.951408 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.951398 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:07.951626 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.951603 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:05:07.953660 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.953611 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-kube-api-access-brchv" (OuterVolumeSpecName: "kube-api-access-brchv") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "kube-api-access-brchv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:05:07.953792 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.953702 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:05:07.953995 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.953942 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:05:07.953995 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:07.953947 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b57c723-7f69-46fb-974b-a10f13d5d3e9" (UID: "5b57c723-7f69-46fb-974b-a10f13d5d3e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:05:08.051840 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.051796 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:08.051840 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.051831 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brchv\" (UniqueName: \"kubernetes.io/projected/5b57c723-7f69-46fb-974b-a10f13d5d3e9-kube-api-access-brchv\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:08.051840 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.051844 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:08.052117 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.051863 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:08.052117 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.051875 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b57c723-7f69-46fb-974b-a10f13d5d3e9-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:05:08.610044 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.610012 2558 generic.go:356] "Generic (PLEG): container finished" podID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerID="c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07" exitCode=0 Sep 07 09:05:08.610240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.610057 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" event={"ID":"5b57c723-7f69-46fb-974b-a10f13d5d3e9","Type":"ContainerDied","Data":"c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07"} Sep 07 09:05:08.610240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.610071 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" Sep 07 09:05:08.610240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.610108 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2" event={"ID":"5b57c723-7f69-46fb-974b-a10f13d5d3e9","Type":"ContainerDied","Data":"7320ffb65d1f75235c8dc21de41f09f086be61d0e6b701028330a3d6fa7a8ace"} Sep 07 09:05:08.610240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.610137 2558 scope.go:117] "RemoveContainer" containerID="c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07" Sep 07 09:05:08.641598 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.641570 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2"] Sep 07 09:05:08.646643 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.646614 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcqqmv2"] Sep 07 09:05:08.814668 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.814544 2558 scope.go:117] "RemoveContainer" containerID="c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07" Sep 07 09:05:08.815007 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:05:08.814974 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07\": container with ID starting with c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07 not found: ID does not exist" containerID="c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07" Sep 07 09:05:08.815132 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.815010 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07"} err="failed to get container status \"c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07\": rpc error: code = NotFound desc = could not find container \"c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07\": container with ID starting with c3e35fe8062b870cb3142ca7f98aa3d530f52fafb45ba68499d29ea7d2c0fa07 not found: ID does not exist" Sep 07 09:05:08.876222 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.876187 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:05:08.876522 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:08.876259 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:05:09.620775 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:09.620737 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" event={"ID":"b24df2d2-d777-47ba-9509-c3b16745fce7","Type":"ContainerStarted","Data":"6faba647521458dace3fa305c3f8553101d6acf13596418575c83f8d7a905367"} Sep 07 09:05:09.621598 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:09.621568 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:09.624020 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:09.623991 2558 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-wxd54 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 07 09:05:09.624138 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:09.624069 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" podUID="b24df2d2-d777-47ba-9509-c3b16745fce7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 07 09:05:09.951009 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:09.950931 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" path="/var/lib/kubelet/pods/5b57c723-7f69-46fb-974b-a10f13d5d3e9/volumes" Sep 07 09:05:10.626416 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:10.626386 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" Sep 07 09:05:10.647991 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:10.647942 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-wxd54" podStartSLOduration=3.284091626 podStartE2EDuration="5.647925399s" totalImagesPullingTime="2.363833773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:05:09.651197152 +0000 UTC m=+368.167988055" watchObservedRunningTime="2026-09-07 09:05:10.647925399 +0000 UTC m=+369.164716302" Sep 07 09:05:15.593849 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:15.593821 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-s5bdn" Sep 07 09:05:15.595784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:15.595760 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lkmxx" Sep 07 09:05:39.925685 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:39.925647 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59cccffb88-7q9qc"] Sep 07 09:05:39.926142 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:39.925994 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerName="istio-proxy" Sep 07 09:05:39.926142 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:39.926006 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerName="istio-proxy" Sep 07 09:05:39.926142 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:39.926066 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b57c723-7f69-46fb-974b-a10f13d5d3e9" containerName="istio-proxy" Sep 07 09:05:39.936365 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:39.936341 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:39.939187 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:39.939162 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cccffb88-7q9qc"] Sep 07 09:05:40.020348 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.020286 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-service-ca\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.020348 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.020345 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc978\" (UniqueName: \"kubernetes.io/projected/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-kube-api-access-fc978\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.020560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.020434 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-trusted-ca-bundle\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.020560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.020461 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-oauth-serving-cert\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.020560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.020487 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-oauth-config\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.020560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.020503 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-config\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.020560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.020549 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-serving-cert\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121115 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121085 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-service-ca\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121115 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc978\" (UniqueName: \"kubernetes.io/projected/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-kube-api-access-fc978\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121376 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121157 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-trusted-ca-bundle\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121376 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121175 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-oauth-serving-cert\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121376 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121208 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-oauth-config\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121376 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121232 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-config\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121376 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-serving-cert\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.121926 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.121897 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-service-ca\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.122072 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.122000 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-oauth-serving-cert\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.122135 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.122062 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-trusted-ca-bundle\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.122135 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.122099 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-config\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.123704 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.123674 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-serving-cert\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.123823 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.123803 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-console-oauth-config\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.128950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.128932 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc978\" (UniqueName: \"kubernetes.io/projected/3f7511e1-8285-4514-bd2a-ba2bf0e4c635-kube-api-access-fc978\") pod \"console-59cccffb88-7q9qc\" (UID: \"3f7511e1-8285-4514-bd2a-ba2bf0e4c635\") " pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.246810 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.246727 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:40.521584 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.521534 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59cccffb88-7q9qc"] Sep 07 09:05:40.524021 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:05:40.523995 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7511e1_8285_4514_bd2a_ba2bf0e4c635.slice/crio-23367bdec554621c33c0ae8ffe3d2cea8f304cdfefe91e5581e72ec5e4834c10 WatchSource:0}: Error finding container 23367bdec554621c33c0ae8ffe3d2cea8f304cdfefe91e5581e72ec5e4834c10: Status 404 returned error can't find the container with id 23367bdec554621c33c0ae8ffe3d2cea8f304cdfefe91e5581e72ec5e4834c10 Sep 07 09:05:40.772047 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.771951 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cccffb88-7q9qc" event={"ID":"3f7511e1-8285-4514-bd2a-ba2bf0e4c635","Type":"ContainerStarted","Data":"e46f233f1f8151cd769916acde31de442464411d13fed0666ef1a8be39b74ea2"} Sep 07 09:05:40.772047 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.771998 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59cccffb88-7q9qc" event={"ID":"3f7511e1-8285-4514-bd2a-ba2bf0e4c635","Type":"ContainerStarted","Data":"23367bdec554621c33c0ae8ffe3d2cea8f304cdfefe91e5581e72ec5e4834c10"} Sep 07 09:05:40.789680 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:40.789634 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59cccffb88-7q9qc" podStartSLOduration=1.789617827 podStartE2EDuration="1.789617827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:05:40.788336234 +0000 UTC m=+399.305127137" watchObservedRunningTime="2026-09-07 09:05:40.789617827 +0000 UTC m=+399.306408729" Sep 07 09:05:50.247581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:50.247537 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:50.247581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:50.247604 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:50.252233 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:50.252201 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:50.829212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:50.829182 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59cccffb88-7q9qc" Sep 07 09:05:50.876323 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:05:50.876268 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67546f76b9-5j685"] Sep 07 09:06:04.361776 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.361736 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl"] Sep 07 09:06:04.366652 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.366627 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.368984 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.368960 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 07 09:06:04.369110 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.369028 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-f5tkk\"" Sep 07 09:06:04.369110 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.368963 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 07 09:06:04.372949 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.372923 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl"] Sep 07 09:06:04.429579 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.429551 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.429748 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.429585 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.429748 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.429728 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xvh9\" (UniqueName: \"kubernetes.io/projected/6002ef89-91ff-4978-ad96-a9f2faa82b72-kube-api-access-4xvh9\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.459707 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.459676 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg"] Sep 07 09:06:04.463136 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.463115 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.474817 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.474790 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg"] Sep 07 09:06:04.530252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530214 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqqxr\" (UniqueName: \"kubernetes.io/projected/188269e7-83d8-4f7f-923f-2e0b3df61a1d-kube-api-access-lqqxr\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.530252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.530484 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530285 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.530484 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530385 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xvh9\" (UniqueName: \"kubernetes.io/projected/6002ef89-91ff-4978-ad96-a9f2faa82b72-kube-api-access-4xvh9\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.530484 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530426 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.530484 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530459 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.530849 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530826 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.530930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.530866 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.546891 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.546861 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xvh9\" (UniqueName: \"kubernetes.io/projected/6002ef89-91ff-4978-ad96-a9f2faa82b72-kube-api-access-4xvh9\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.606380 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.606349 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9"] Sep 07 09:06:04.610062 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.610044 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.617987 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.617961 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9"] Sep 07 09:06:04.631269 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631241 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.631398 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631277 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjbtt\" (UniqueName: \"kubernetes.io/projected/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-kube-api-access-wjbtt\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.631398 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631317 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.631470 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631418 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.631470 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631457 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqqxr\" (UniqueName: \"kubernetes.io/projected/188269e7-83d8-4f7f-923f-2e0b3df61a1d-kube-api-access-lqqxr\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.631539 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.631674 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631656 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.631754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.631730 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.639557 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.639530 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqqxr\" (UniqueName: \"kubernetes.io/projected/188269e7-83d8-4f7f-923f-2e0b3df61a1d-kube-api-access-lqqxr\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.668662 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.668634 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z"] Sep 07 09:06:04.672141 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.672123 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.676606 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.676585 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:04.681036 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.681010 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z"] Sep 07 09:06:04.732423 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.732390 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.732551 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.732447 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d277m\" (UniqueName: \"kubernetes.io/projected/189a8bf3-0494-4182-8a4a-8e49a817f18c-kube-api-access-d277m\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.732551 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.732527 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjbtt\" (UniqueName: \"kubernetes.io/projected/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-kube-api-access-wjbtt\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.732667 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.732555 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.732667 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.732643 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.732789 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.732672 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.733286 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.733262 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.733430 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.733408 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.741685 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.741630 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjbtt\" (UniqueName: \"kubernetes.io/projected/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-kube-api-access-wjbtt\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.772023 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.771992 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:04.833871 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.833840 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.834009 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.833877 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.834009 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.833899 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d277m\" (UniqueName: \"kubernetes.io/projected/189a8bf3-0494-4182-8a4a-8e49a817f18c-kube-api-access-d277m\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.834264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.834242 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.834349 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.834281 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.842112 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.842087 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d277m\" (UniqueName: \"kubernetes.io/projected/189a8bf3-0494-4182-8a4a-8e49a817f18c-kube-api-access-d277m\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:04.920844 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.920753 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:04.952512 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.952486 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl"] Sep 07 09:06:04.953940 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:06:04.953909 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6002ef89_91ff_4978_ad96_a9f2faa82b72.slice/crio-074684533ddf21cbe96629475689ed56731a2669e9c201fb1c936b2229d9bcf8 WatchSource:0}: Error finding container 074684533ddf21cbe96629475689ed56731a2669e9c201fb1c936b2229d9bcf8: Status 404 returned error can't find the container with id 074684533ddf21cbe96629475689ed56731a2669e9c201fb1c936b2229d9bcf8 Sep 07 09:06:04.983075 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:04.983050 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:05.049832 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.049766 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg"] Sep 07 09:06:05.051554 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:06:05.051528 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188269e7_83d8_4f7f_923f_2e0b3df61a1d.slice/crio-06a6b57145da6f20e8a9f77709b6ede4b08f100d63f1e8bbc90b0b16ba0103df WatchSource:0}: Error finding container 06a6b57145da6f20e8a9f77709b6ede4b08f100d63f1e8bbc90b0b16ba0103df: Status 404 returned error can't find the container with id 06a6b57145da6f20e8a9f77709b6ede4b08f100d63f1e8bbc90b0b16ba0103df Sep 07 09:06:05.195729 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.195702 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9"] Sep 07 09:06:05.197352 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:06:05.197319 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1e8d37_0d1b_4cd4_80d0_22aa0a4decbb.slice/crio-f2dac952b20b05fab2c1c8bf18db4c52e7c58294dccbc074b20ad6397b0407fe WatchSource:0}: Error finding container f2dac952b20b05fab2c1c8bf18db4c52e7c58294dccbc074b20ad6397b0407fe: Status 404 returned error can't find the container with id f2dac952b20b05fab2c1c8bf18db4c52e7c58294dccbc074b20ad6397b0407fe Sep 07 09:06:05.257754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.257726 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z"] Sep 07 09:06:05.273867 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:06:05.273835 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189a8bf3_0494_4182_8a4a_8e49a817f18c.slice/crio-78a6cb2d3cd74ca05aff1fa0a7e5c9729e43ac8fcff62029561f5bbd47d3560c WatchSource:0}: Error finding container 78a6cb2d3cd74ca05aff1fa0a7e5c9729e43ac8fcff62029561f5bbd47d3560c: Status 404 returned error can't find the container with id 78a6cb2d3cd74ca05aff1fa0a7e5c9729e43ac8fcff62029561f5bbd47d3560c Sep 07 09:06:05.895825 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.895792 2558 generic.go:356] "Generic (PLEG): container finished" podID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerID="bf65def922be38dc14b2ff411b347689f7e82086daa95eaa5c4523bc3fb46035" exitCode=0 Sep 07 09:06:05.896271 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.895869 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" event={"ID":"189a8bf3-0494-4182-8a4a-8e49a817f18c","Type":"ContainerDied","Data":"bf65def922be38dc14b2ff411b347689f7e82086daa95eaa5c4523bc3fb46035"} Sep 07 09:06:05.896271 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.895917 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" event={"ID":"189a8bf3-0494-4182-8a4a-8e49a817f18c","Type":"ContainerStarted","Data":"78a6cb2d3cd74ca05aff1fa0a7e5c9729e43ac8fcff62029561f5bbd47d3560c"} Sep 07 09:06:05.897542 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.897517 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerID="a0d08f83b525944005f3fcf595cc871111afa364a8ade4186c77eeb6166fb78a" exitCode=0 Sep 07 09:06:05.897641 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.897608 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" event={"ID":"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb","Type":"ContainerDied","Data":"a0d08f83b525944005f3fcf595cc871111afa364a8ade4186c77eeb6166fb78a"} Sep 07 09:06:05.897708 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.897654 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" event={"ID":"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb","Type":"ContainerStarted","Data":"f2dac952b20b05fab2c1c8bf18db4c52e7c58294dccbc074b20ad6397b0407fe"} Sep 07 09:06:05.899187 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.899166 2558 generic.go:356] "Generic (PLEG): container finished" podID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerID="bd331df7760d7a895967ea1963711d813d5926d7afbaf048e115f9f394aca57b" exitCode=0 Sep 07 09:06:05.899273 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.899248 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" event={"ID":"188269e7-83d8-4f7f-923f-2e0b3df61a1d","Type":"ContainerDied","Data":"bd331df7760d7a895967ea1963711d813d5926d7afbaf048e115f9f394aca57b"} Sep 07 09:06:05.899360 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.899292 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" event={"ID":"188269e7-83d8-4f7f-923f-2e0b3df61a1d","Type":"ContainerStarted","Data":"06a6b57145da6f20e8a9f77709b6ede4b08f100d63f1e8bbc90b0b16ba0103df"} Sep 07 09:06:05.900885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.900865 2558 generic.go:356] "Generic (PLEG): container finished" podID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerID="737fd983272bfe6cfb3a54d48f478018c93c2973e8958c27af1d0835975aa33e" exitCode=0 Sep 07 09:06:05.900979 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.900942 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" event={"ID":"6002ef89-91ff-4978-ad96-a9f2faa82b72","Type":"ContainerDied","Data":"737fd983272bfe6cfb3a54d48f478018c93c2973e8958c27af1d0835975aa33e"} Sep 07 09:06:05.901047 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:05.900978 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" event={"ID":"6002ef89-91ff-4978-ad96-a9f2faa82b72","Type":"ContainerStarted","Data":"074684533ddf21cbe96629475689ed56731a2669e9c201fb1c936b2229d9bcf8"} Sep 07 09:06:07.915850 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.915816 2558 generic.go:356] "Generic (PLEG): container finished" podID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerID="8f4aa02ddbacbc2e3e8427bd8b27cb3678ed730922d02096fcba33317c066d82" exitCode=0 Sep 07 09:06:07.916347 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.915903 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" event={"ID":"189a8bf3-0494-4182-8a4a-8e49a817f18c","Type":"ContainerDied","Data":"8f4aa02ddbacbc2e3e8427bd8b27cb3678ed730922d02096fcba33317c066d82"} Sep 07 09:06:07.917687 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.917662 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerID="7844cd000f17e6fa24cb5ac046c35594d4af3be1e47cf3a14c8f91d069a79db4" exitCode=0 Sep 07 09:06:07.917779 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.917725 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" event={"ID":"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb","Type":"ContainerDied","Data":"7844cd000f17e6fa24cb5ac046c35594d4af3be1e47cf3a14c8f91d069a79db4"} Sep 07 09:06:07.919616 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.919598 2558 generic.go:356] "Generic (PLEG): container finished" podID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerID="eeb8cc6ea760d4c521aeb128140c68697c1b4b3381f02bbe6999660e40eca6d3" exitCode=0 Sep 07 09:06:07.919696 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.919679 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" event={"ID":"188269e7-83d8-4f7f-923f-2e0b3df61a1d","Type":"ContainerDied","Data":"eeb8cc6ea760d4c521aeb128140c68697c1b4b3381f02bbe6999660e40eca6d3"} Sep 07 09:06:07.921562 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.921543 2558 generic.go:356] "Generic (PLEG): container finished" podID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerID="91aa5a749aa6f3bc80c4761d070711420afcce6a05b68df3019b96afda697927" exitCode=0 Sep 07 09:06:07.921631 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:07.921616 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" event={"ID":"6002ef89-91ff-4978-ad96-a9f2faa82b72","Type":"ContainerDied","Data":"91aa5a749aa6f3bc80c4761d070711420afcce6a05b68df3019b96afda697927"} Sep 07 09:06:08.930428 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.930395 2558 generic.go:356] "Generic (PLEG): container finished" podID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerID="e02d076f7f984a64d645bff50204de1d3c46171cd7a627f3bf4dc17e97616f2f" exitCode=0 Sep 07 09:06:08.930833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.930469 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" event={"ID":"189a8bf3-0494-4182-8a4a-8e49a817f18c","Type":"ContainerDied","Data":"e02d076f7f984a64d645bff50204de1d3c46171cd7a627f3bf4dc17e97616f2f"} Sep 07 09:06:08.932654 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.932629 2558 generic.go:356] "Generic (PLEG): container finished" podID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerID="370e9172ba70f6f04bb7b433a44cd18a0963c659984d5ede48c97d7a9da4e3ac" exitCode=0 Sep 07 09:06:08.932774 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.932711 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" event={"ID":"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb","Type":"ContainerDied","Data":"370e9172ba70f6f04bb7b433a44cd18a0963c659984d5ede48c97d7a9da4e3ac"} Sep 07 09:06:08.934871 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.934851 2558 generic.go:356] "Generic (PLEG): container finished" podID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerID="ee848a252383f4a85db8a3b986992c10198a8536020c10d42022ff6120f5df1c" exitCode=0 Sep 07 09:06:08.934971 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.934922 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" event={"ID":"188269e7-83d8-4f7f-923f-2e0b3df61a1d","Type":"ContainerDied","Data":"ee848a252383f4a85db8a3b986992c10198a8536020c10d42022ff6120f5df1c"} Sep 07 09:06:08.936927 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.936908 2558 generic.go:356] "Generic (PLEG): container finished" podID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerID="d67feed064c568e858d1c70722cbac3340f44791ffa65e200ddc40fb07818e33" exitCode=0 Sep 07 09:06:08.937008 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:08.936988 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" event={"ID":"6002ef89-91ff-4978-ad96-a9f2faa82b72","Type":"ContainerDied","Data":"d67feed064c568e858d1c70722cbac3340f44791ffa65e200ddc40fb07818e33"} Sep 07 09:06:10.115735 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.115711 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:10.119509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.119488 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:10.139725 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.139704 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:10.143591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.143564 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:10.179741 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179712 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d277m\" (UniqueName: \"kubernetes.io/projected/189a8bf3-0494-4182-8a4a-8e49a817f18c-kube-api-access-d277m\") pod \"189a8bf3-0494-4182-8a4a-8e49a817f18c\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " Sep 07 09:06:10.179741 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179749 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-bundle\") pod \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " Sep 07 09:06:10.179961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179775 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-bundle\") pod \"189a8bf3-0494-4182-8a4a-8e49a817f18c\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " Sep 07 09:06:10.179961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179801 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-bundle\") pod \"6002ef89-91ff-4978-ad96-a9f2faa82b72\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " Sep 07 09:06:10.179961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179847 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-util\") pod \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " Sep 07 09:06:10.179961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179870 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-util\") pod \"6002ef89-91ff-4978-ad96-a9f2faa82b72\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " Sep 07 09:06:10.179961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179898 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-bundle\") pod \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " Sep 07 09:06:10.179961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179944 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xvh9\" (UniqueName: \"kubernetes.io/projected/6002ef89-91ff-4978-ad96-a9f2faa82b72-kube-api-access-4xvh9\") pod \"6002ef89-91ff-4978-ad96-a9f2faa82b72\" (UID: \"6002ef89-91ff-4978-ad96-a9f2faa82b72\") " Sep 07 09:06:10.180252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.179999 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-util\") pod \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " Sep 07 09:06:10.180252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.180030 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-util\") pod \"189a8bf3-0494-4182-8a4a-8e49a817f18c\" (UID: \"189a8bf3-0494-4182-8a4a-8e49a817f18c\") " Sep 07 09:06:10.180252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.180068 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjbtt\" (UniqueName: \"kubernetes.io/projected/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-kube-api-access-wjbtt\") pod \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\" (UID: \"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb\") " Sep 07 09:06:10.180252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.180097 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqqxr\" (UniqueName: \"kubernetes.io/projected/188269e7-83d8-4f7f-923f-2e0b3df61a1d-kube-api-access-lqqxr\") pod \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\" (UID: \"188269e7-83d8-4f7f-923f-2e0b3df61a1d\") " Sep 07 09:06:10.180890 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.180379 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-bundle" (OuterVolumeSpecName: "bundle") pod "ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" (UID: "ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.180890 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.180553 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-bundle" (OuterVolumeSpecName: "bundle") pod "189a8bf3-0494-4182-8a4a-8e49a817f18c" (UID: "189a8bf3-0494-4182-8a4a-8e49a817f18c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.181075 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.180974 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-bundle" (OuterVolumeSpecName: "bundle") pod "6002ef89-91ff-4978-ad96-a9f2faa82b72" (UID: "6002ef89-91ff-4978-ad96-a9f2faa82b72"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.181075 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.181020 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-bundle" (OuterVolumeSpecName: "bundle") pod "188269e7-83d8-4f7f-923f-2e0b3df61a1d" (UID: "188269e7-83d8-4f7f-923f-2e0b3df61a1d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.182660 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.182630 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189a8bf3-0494-4182-8a4a-8e49a817f18c-kube-api-access-d277m" (OuterVolumeSpecName: "kube-api-access-d277m") pod "189a8bf3-0494-4182-8a4a-8e49a817f18c" (UID: "189a8bf3-0494-4182-8a4a-8e49a817f18c"). InnerVolumeSpecName "kube-api-access-d277m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:06:10.183729 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.182853 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/188269e7-83d8-4f7f-923f-2e0b3df61a1d-kube-api-access-lqqxr" (OuterVolumeSpecName: "kube-api-access-lqqxr") pod "188269e7-83d8-4f7f-923f-2e0b3df61a1d" (UID: "188269e7-83d8-4f7f-923f-2e0b3df61a1d"). InnerVolumeSpecName "kube-api-access-lqqxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:06:10.183729 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.183163 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6002ef89-91ff-4978-ad96-a9f2faa82b72-kube-api-access-4xvh9" (OuterVolumeSpecName: "kube-api-access-4xvh9") pod "6002ef89-91ff-4978-ad96-a9f2faa82b72" (UID: "6002ef89-91ff-4978-ad96-a9f2faa82b72"). InnerVolumeSpecName "kube-api-access-4xvh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:06:10.184454 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.184423 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-kube-api-access-wjbtt" (OuterVolumeSpecName: "kube-api-access-wjbtt") pod "ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" (UID: "ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb"). InnerVolumeSpecName "kube-api-access-wjbtt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:06:10.189399 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.189376 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-util" (OuterVolumeSpecName: "util") pod "189a8bf3-0494-4182-8a4a-8e49a817f18c" (UID: "189a8bf3-0494-4182-8a4a-8e49a817f18c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.189605 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.189586 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-util" (OuterVolumeSpecName: "util") pod "ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" (UID: "ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.190837 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.190370 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-util" (OuterVolumeSpecName: "util") pod "188269e7-83d8-4f7f-923f-2e0b3df61a1d" (UID: "188269e7-83d8-4f7f-923f-2e0b3df61a1d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.223886 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.223801 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-util" (OuterVolumeSpecName: "util") pod "6002ef89-91ff-4978-ad96-a9f2faa82b72" (UID: "6002ef89-91ff-4978-ad96-a9f2faa82b72"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:06:10.281501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281467 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281496 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjbtt\" (UniqueName: \"kubernetes.io/projected/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-kube-api-access-wjbtt\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281507 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqqxr\" (UniqueName: \"kubernetes.io/projected/188269e7-83d8-4f7f-923f-2e0b3df61a1d-kube-api-access-lqqxr\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281516 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d277m\" (UniqueName: \"kubernetes.io/projected/189a8bf3-0494-4182-8a4a-8e49a817f18c-kube-api-access-d277m\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281525 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281532 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/189a8bf3-0494-4182-8a4a-8e49a817f18c-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281540 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281547 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281554 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6002ef89-91ff-4978-ad96-a9f2faa82b72-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281561 2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281569 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xvh9\" (UniqueName: \"kubernetes.io/projected/6002ef89-91ff-4978-ad96-a9f2faa82b72-kube-api-access-4xvh9\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.281713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.281576 2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/188269e7-83d8-4f7f-923f-2e0b3df61a1d-util\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:10.950140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.950116 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" Sep 07 09:06:10.950421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.950115 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bblp92z" event={"ID":"189a8bf3-0494-4182-8a4a-8e49a817f18c","Type":"ContainerDied","Data":"78a6cb2d3cd74ca05aff1fa0a7e5c9729e43ac8fcff62029561f5bbd47d3560c"} Sep 07 09:06:10.950421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.950226 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78a6cb2d3cd74ca05aff1fa0a7e5c9729e43ac8fcff62029561f5bbd47d3560c" Sep 07 09:06:10.952173 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.952151 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" Sep 07 09:06:10.952334 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.952183 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0svwb9" event={"ID":"ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb","Type":"ContainerDied","Data":"f2dac952b20b05fab2c1c8bf18db4c52e7c58294dccbc074b20ad6397b0407fe"} Sep 07 09:06:10.952334 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.952220 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2dac952b20b05fab2c1c8bf18db4c52e7c58294dccbc074b20ad6397b0407fe" Sep 07 09:06:10.954214 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.954189 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" Sep 07 09:06:10.954345 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.954215 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183ccswtg" event={"ID":"188269e7-83d8-4f7f-923f-2e0b3df61a1d","Type":"ContainerDied","Data":"06a6b57145da6f20e8a9f77709b6ede4b08f100d63f1e8bbc90b0b16ba0103df"} Sep 07 09:06:10.954345 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.954252 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06a6b57145da6f20e8a9f77709b6ede4b08f100d63f1e8bbc90b0b16ba0103df" Sep 07 09:06:10.956405 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.956380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" event={"ID":"6002ef89-91ff-4978-ad96-a9f2faa82b72","Type":"ContainerDied","Data":"074684533ddf21cbe96629475689ed56731a2669e9c201fb1c936b2229d9bcf8"} Sep 07 09:06:10.956519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.956403 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053xlbgl" Sep 07 09:06:10.956519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:10.956420 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="074684533ddf21cbe96629475689ed56731a2669e9c201fb1c936b2229d9bcf8" Sep 07 09:06:15.902621 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:15.902556 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67546f76b9-5j685" podUID="7f9d2657-61a9-4149-93d1-4373effdc096" containerName="console" containerID="cri-o://abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417" gracePeriod=15 Sep 07 09:06:16.147844 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.147819 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:06:16.236536 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236449 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-console-config\") pod \"7f9d2657-61a9-4149-93d1-4373effdc096\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " Sep 07 09:06:16.236536 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236511 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-oauth-config\") pod \"7f9d2657-61a9-4149-93d1-4373effdc096\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " Sep 07 09:06:16.236760 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236540 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-service-ca\") pod \"7f9d2657-61a9-4149-93d1-4373effdc096\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " Sep 07 09:06:16.236760 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236570 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-oauth-serving-cert\") pod \"7f9d2657-61a9-4149-93d1-4373effdc096\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " Sep 07 09:06:16.236760 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236643 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-trusted-ca-bundle\") pod \"7f9d2657-61a9-4149-93d1-4373effdc096\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " Sep 07 09:06:16.236760 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236672 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-serving-cert\") pod \"7f9d2657-61a9-4149-93d1-4373effdc096\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " Sep 07 09:06:16.236760 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236717 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcbph\" (UniqueName: \"kubernetes.io/projected/7f9d2657-61a9-4149-93d1-4373effdc096-kube-api-access-vcbph\") pod \"7f9d2657-61a9-4149-93d1-4373effdc096\" (UID: \"7f9d2657-61a9-4149-93d1-4373effdc096\") " Sep 07 09:06:16.237030 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.236998 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-console-config" (OuterVolumeSpecName: "console-config") pod "7f9d2657-61a9-4149-93d1-4373effdc096" (UID: "7f9d2657-61a9-4149-93d1-4373effdc096"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:06:16.237088 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.237045 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7f9d2657-61a9-4149-93d1-4373effdc096" (UID: "7f9d2657-61a9-4149-93d1-4373effdc096"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:06:16.237088 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.237054 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-service-ca" (OuterVolumeSpecName: "service-ca") pod "7f9d2657-61a9-4149-93d1-4373effdc096" (UID: "7f9d2657-61a9-4149-93d1-4373effdc096"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:06:16.237169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.237130 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7f9d2657-61a9-4149-93d1-4373effdc096" (UID: "7f9d2657-61a9-4149-93d1-4373effdc096"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:06:16.238788 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.238754 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7f9d2657-61a9-4149-93d1-4373effdc096" (UID: "7f9d2657-61a9-4149-93d1-4373effdc096"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:06:16.238885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.238823 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7f9d2657-61a9-4149-93d1-4373effdc096" (UID: "7f9d2657-61a9-4149-93d1-4373effdc096"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:06:16.238885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.238878 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f9d2657-61a9-4149-93d1-4373effdc096-kube-api-access-vcbph" (OuterVolumeSpecName: "kube-api-access-vcbph") pod "7f9d2657-61a9-4149-93d1-4373effdc096" (UID: "7f9d2657-61a9-4149-93d1-4373effdc096"). InnerVolumeSpecName "kube-api-access-vcbph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:06:16.337674 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.337634 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-trusted-ca-bundle\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:16.337674 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.337668 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:16.337674 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.337677 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcbph\" (UniqueName: \"kubernetes.io/projected/7f9d2657-61a9-4149-93d1-4373effdc096-kube-api-access-vcbph\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:16.337901 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.337687 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-console-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:16.337901 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.337696 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f9d2657-61a9-4149-93d1-4373effdc096-console-oauth-config\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:16.337901 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.337705 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:16.337901 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.337712 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f9d2657-61a9-4149-93d1-4373effdc096-oauth-serving-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:06:16.986619 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.986584 2558 generic.go:356] "Generic (PLEG): container finished" podID="7f9d2657-61a9-4149-93d1-4373effdc096" containerID="abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417" exitCode=0 Sep 07 09:06:16.986619 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.986622 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67546f76b9-5j685" event={"ID":"7f9d2657-61a9-4149-93d1-4373effdc096","Type":"ContainerDied","Data":"abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417"} Sep 07 09:06:16.987051 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.986630 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67546f76b9-5j685" Sep 07 09:06:16.987051 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.986654 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67546f76b9-5j685" event={"ID":"7f9d2657-61a9-4149-93d1-4373effdc096","Type":"ContainerDied","Data":"de1f6ff1024f4d1d60f0d9dacae021ec6aea612d553e8f1d4464e1ead31b4f64"} Sep 07 09:06:16.987051 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.986675 2558 scope.go:117] "RemoveContainer" containerID="abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417" Sep 07 09:06:16.995784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.995767 2558 scope.go:117] "RemoveContainer" containerID="abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417" Sep 07 09:06:16.996030 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:06:16.996010 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417\": container with ID starting with abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417 not found: ID does not exist" containerID="abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417" Sep 07 09:06:16.996110 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:16.996049 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417"} err="failed to get container status \"abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417\": rpc error: code = NotFound desc = could not find container \"abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417\": container with ID starting with abbb9b85d0704e2da6ea7f3c3fa02bce95cc0392f2a994b3b996d37915f43417 not found: ID does not exist" Sep 07 09:06:17.008804 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:17.008781 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67546f76b9-5j685"] Sep 07 09:06:17.015065 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:17.015045 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67546f76b9-5j685"] Sep 07 09:06:17.950900 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:06:17.950866 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f9d2657-61a9-4149-93d1-4373effdc096" path="/var/lib/kubelet/pods/7f9d2657-61a9-4149-93d1-4373effdc096/volumes" Sep 07 09:07:24.878196 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878165 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc"] Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878510 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878527 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878548 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878553 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878560 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878566 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878574 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878579 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878588 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f9d2657-61a9-4149-93d1-4373effdc096" containerName="console" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878594 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f9d2657-61a9-4149-93d1-4373effdc096" containerName="console" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878600 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerName="pull" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878608 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerName="pull" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878619 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878627 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878638 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerName="pull" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878646 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerName="pull" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878656 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878661 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878666 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878671 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerName="util" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878677 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerName="pull" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878682 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerName="pull" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878687 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878692 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerName="extract" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878699 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerName="pull" Sep 07 09:07:24.878692 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878708 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerName="pull" Sep 07 09:07:24.879548 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878764 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1e8d37-0d1b-4cd4-80d0-22aa0a4decbb" containerName="extract" Sep 07 09:07:24.879548 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878772 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="188269e7-83d8-4f7f-923f-2e0b3df61a1d" containerName="extract" Sep 07 09:07:24.879548 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878779 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f9d2657-61a9-4149-93d1-4373effdc096" containerName="console" Sep 07 09:07:24.879548 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878787 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="189a8bf3-0494-4182-8a4a-8e49a817f18c" containerName="extract" Sep 07 09:07:24.879548 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.878797 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="6002ef89-91ff-4978-ad96-a9f2faa82b72" containerName="extract" Sep 07 09:07:24.882827 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.882799 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:24.885375 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.885355 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qww9f\"" Sep 07 09:07:24.893594 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:24.893571 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc"] Sep 07 09:07:25.014092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014061 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014100 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014122 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014189 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014232 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzxgd\" (UniqueName: \"kubernetes.io/projected/3b0b6217-00aa-46ef-a29f-55ee000b5152-kube-api-access-bzxgd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014424 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014252 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014424 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014267 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014424 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014360 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b6217-00aa-46ef-a29f-55ee000b5152-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.014424 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.014389 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115347 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115286 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b6217-00aa-46ef-a29f-55ee000b5152-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115347 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115563 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115376 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115563 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115406 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115563 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115428 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115563 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115447 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115746 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115579 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzxgd\" (UniqueName: \"kubernetes.io/projected/3b0b6217-00aa-46ef-a29f-55ee000b5152-kube-api-access-bzxgd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115868 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115847 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115899 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.115950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115936 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.116067 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.115976 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.116128 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.116107 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b0b6217-00aa-46ef-a29f-55ee000b5152-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.116199 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.116174 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.116331 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.116315 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.117761 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.117739 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.118234 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.118218 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.123706 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.123676 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b0b6217-00aa-46ef-a29f-55ee000b5152-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.123966 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.123945 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzxgd\" (UniqueName: \"kubernetes.io/projected/3b0b6217-00aa-46ef-a29f-55ee000b5152-kube-api-access-bzxgd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ck6nc\" (UID: \"3b0b6217-00aa-46ef-a29f-55ee000b5152\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.194195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.194099 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:25.470891 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.470864 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc"] Sep 07 09:07:25.472698 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:07:25.472666 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b0b6217_00aa_46ef_a29f_55ee000b5152.slice/crio-bed7f1551697191df795930bf7cec70e11b25662479c2add4e26a46b832ec683 WatchSource:0}: Error finding container bed7f1551697191df795930bf7cec70e11b25662479c2add4e26a46b832ec683: Status 404 returned error can't find the container with id bed7f1551697191df795930bf7cec70e11b25662479c2add4e26a46b832ec683 Sep 07 09:07:25.474795 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.474758 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:07:25.474912 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.474838 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:07:25.474912 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:25.474880 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:07:26.329134 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:26.329097 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" event={"ID":"3b0b6217-00aa-46ef-a29f-55ee000b5152","Type":"ContainerStarted","Data":"2262ae9b9faedbd6b19e8d2ad123cc77306fe85108cbd637815596898e468941"} Sep 07 09:07:26.329545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:26.329148 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" event={"ID":"3b0b6217-00aa-46ef-a29f-55ee000b5152","Type":"ContainerStarted","Data":"bed7f1551697191df795930bf7cec70e11b25662479c2add4e26a46b832ec683"} Sep 07 09:07:26.350546 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:26.350497 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" podStartSLOduration=2.350482169 podStartE2EDuration="2.350482169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:07:26.348506978 +0000 UTC m=+504.865297892" watchObservedRunningTime="2026-09-07 09:07:26.350482169 +0000 UTC m=+504.867273154" Sep 07 09:07:27.195100 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:27.195066 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:27.200008 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:27.199983 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:27.334451 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:27.334423 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:07:27.335412 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:07:27.335391 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:08:59.685072 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.685041 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg"] Sep 07 09:08:59.688128 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.688112 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:08:59.690881 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.690860 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 07 09:08:59.691017 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.690866 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w7cxx\"" Sep 07 09:08:59.699474 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.699451 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg"] Sep 07 09:08:59.707529 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.707506 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbhg5\" (UniqueName: \"kubernetes.io/projected/8442be03-cf12-40ef-94b2-d45c67ac2e8a-kube-api-access-hbhg5\") pod \"payload-pre-processing-59fbc47bf8-q4xpg\" (UID: \"8442be03-cf12-40ef-94b2-d45c67ac2e8a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:08:59.707623 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.707606 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8442be03-cf12-40ef-94b2-d45c67ac2e8a-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-q4xpg\" (UID: \"8442be03-cf12-40ef-94b2-d45c67ac2e8a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:08:59.808486 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.808449 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbhg5\" (UniqueName: \"kubernetes.io/projected/8442be03-cf12-40ef-94b2-d45c67ac2e8a-kube-api-access-hbhg5\") pod \"payload-pre-processing-59fbc47bf8-q4xpg\" (UID: \"8442be03-cf12-40ef-94b2-d45c67ac2e8a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:08:59.808648 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.808550 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8442be03-cf12-40ef-94b2-d45c67ac2e8a-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-q4xpg\" (UID: \"8442be03-cf12-40ef-94b2-d45c67ac2e8a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:08:59.809212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.809185 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8442be03-cf12-40ef-94b2-d45c67ac2e8a-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-q4xpg\" (UID: \"8442be03-cf12-40ef-94b2-d45c67ac2e8a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:08:59.817277 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.817250 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbhg5\" (UniqueName: \"kubernetes.io/projected/8442be03-cf12-40ef-94b2-d45c67ac2e8a-kube-api-access-hbhg5\") pod \"payload-pre-processing-59fbc47bf8-q4xpg\" (UID: \"8442be03-cf12-40ef-94b2-d45c67ac2e8a\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:08:59.999552 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:08:59.999476 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:09:00.268720 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:00.268639 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg"] Sep 07 09:09:00.272229 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:09:00.272199 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8442be03_cf12_40ef_94b2_d45c67ac2e8a.slice/crio-c9b5d1e54df47ac1c5960286f03891bd0bd316f41e4c1a20e40ddf2c896c5af6 WatchSource:0}: Error finding container c9b5d1e54df47ac1c5960286f03891bd0bd316f41e4c1a20e40ddf2c896c5af6: Status 404 returned error can't find the container with id c9b5d1e54df47ac1c5960286f03891bd0bd316f41e4c1a20e40ddf2c896c5af6 Sep 07 09:09:00.806531 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:00.806496 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" event={"ID":"8442be03-cf12-40ef-94b2-d45c67ac2e8a","Type":"ContainerStarted","Data":"c9b5d1e54df47ac1c5960286f03891bd0bd316f41e4c1a20e40ddf2c896c5af6"} Sep 07 09:09:01.896827 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:01.896799 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:09:01.897816 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:01.897796 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:09:02.821990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:02.821954 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" event={"ID":"8442be03-cf12-40ef-94b2-d45c67ac2e8a","Type":"ContainerStarted","Data":"6cfa9f186b363da689f4b522283f0ae09377f6baa4a0a238532b6b5523e956f0"} Sep 07 09:09:02.822894 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:02.822866 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:09:08.860173 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:08.860095 2558 generic.go:356] "Generic (PLEG): container finished" podID="3b0b6217-00aa-46ef-a29f-55ee000b5152" containerID="2262ae9b9faedbd6b19e8d2ad123cc77306fe85108cbd637815596898e468941" exitCode=0 Sep 07 09:09:08.860587 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:08.860163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" event={"ID":"3b0b6217-00aa-46ef-a29f-55ee000b5152","Type":"ContainerDied","Data":"2262ae9b9faedbd6b19e8d2ad123cc77306fe85108cbd637815596898e468941"} Sep 07 09:09:08.860587 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:08.860574 2558 scope.go:117] "RemoveContainer" containerID="2262ae9b9faedbd6b19e8d2ad123cc77306fe85108cbd637815596898e468941" Sep 07 09:09:08.861577 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:08.861550 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:09:08.861644 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:08.861616 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:09:08.861685 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:08.861646 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:09:08.881651 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:08.881604 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" podStartSLOduration=7.770872368 podStartE2EDuration="9.881591593s" totalImagesPullingTime="2.110719225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:09:02.842379225 +0000 UTC m=+601.359170129" watchObservedRunningTime="2026-09-07 09:09:08.881591593 +0000 UTC m=+607.398382496" Sep 07 09:09:09.867659 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:09.867620 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" event={"ID":"3b0b6217-00aa-46ef-a29f-55ee000b5152","Type":"ContainerStarted","Data":"3fd05460bdce7dc474d5377099d101acefcbb14c8e2c78cf71cfdb1163e437af"} Sep 07 09:09:10.194452 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:10.194366 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:09:10.195586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:10.195560 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck6nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 07 09:09:10.195684 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:10.195613 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" podUID="3b0b6217-00aa-46ef-a29f-55ee000b5152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 07 09:09:11.195284 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:11.195248 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck6nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 07 09:09:11.195667 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:11.195332 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" podUID="3b0b6217-00aa-46ef-a29f-55ee000b5152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 07 09:09:12.194868 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:12.194828 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck6nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 07 09:09:12.195058 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:12.194898 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" podUID="3b0b6217-00aa-46ef-a29f-55ee000b5152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 07 09:09:12.335289 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:12.335254 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:09:13.195267 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:13.195233 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck6nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 07 09:09:13.195473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:13.195289 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" podUID="3b0b6217-00aa-46ef-a29f-55ee000b5152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 07 09:09:14.194525 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:14.194489 2558 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ck6nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Sep 07 09:09:14.194909 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:14.194544 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" podUID="3b0b6217-00aa-46ef-a29f-55ee000b5152" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Sep 07 09:09:14.840024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:14.839979 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-q4xpg" Sep 07 09:09:15.199084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:15.199000 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:09:15.200022 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:15.200002 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ck6nc" Sep 07 09:09:44.298860 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.298819 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn"] Sep 07 09:09:44.302475 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.302454 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.306033 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.306011 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 07 09:09:44.306148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.306014 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 07 09:09:44.306148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.306015 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 07 09:09:44.306148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.306064 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5tt6q\"" Sep 07 09:09:44.312856 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.312835 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn"] Sep 07 09:09:44.320370 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.320351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.320490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.320380 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.320490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.320407 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.320490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.320452 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.320630 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.320515 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nllct\" (UniqueName: \"kubernetes.io/projected/acda191f-8162-43fe-af31-aeaad0d39422-kube-api-access-nllct\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.320630 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.320552 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acda191f-8162-43fe-af31-aeaad0d39422-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.421286 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.421252 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.421489 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.421316 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.421489 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.421352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nllct\" (UniqueName: \"kubernetes.io/projected/acda191f-8162-43fe-af31-aeaad0d39422-kube-api-access-nllct\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.421489 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.421393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acda191f-8162-43fe-af31-aeaad0d39422-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.428569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.421830 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.428569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.421900 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.428569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.421947 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.428569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.422197 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.428569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.422263 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.428569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.424397 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acda191f-8162-43fe-af31-aeaad0d39422-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.428569 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.424837 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acda191f-8162-43fe-af31-aeaad0d39422-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.430638 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.430613 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nllct\" (UniqueName: \"kubernetes.io/projected/acda191f-8162-43fe-af31-aeaad0d39422-kube-api-access-nllct\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pndkn\" (UID: \"acda191f-8162-43fe-af31-aeaad0d39422\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.613873 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.613791 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:44.885967 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.885894 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn"] Sep 07 09:09:44.888813 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:09:44.888785 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacda191f_8162_43fe_af31_aeaad0d39422.slice/crio-f3a070cbed604c20c6592cb7e7d51d407f447f6e46d748b45c97e82258ac5c1a WatchSource:0}: Error finding container f3a070cbed604c20c6592cb7e7d51d407f447f6e46d748b45c97e82258ac5c1a: Status 404 returned error can't find the container with id f3a070cbed604c20c6592cb7e7d51d407f447f6e46d748b45c97e82258ac5c1a Sep 07 09:09:44.890158 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:44.890140 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 07 09:09:45.064570 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:45.064535 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" event={"ID":"acda191f-8162-43fe-af31-aeaad0d39422","Type":"ContainerStarted","Data":"f3a070cbed604c20c6592cb7e7d51d407f447f6e46d748b45c97e82258ac5c1a"} Sep 07 09:09:46.071659 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:46.071625 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" event={"ID":"acda191f-8162-43fe-af31-aeaad0d39422","Type":"ContainerStarted","Data":"e5da892d47104a59ed48fa807b2fd11c3abbec3186827a44a061c97dcda7fb31"} Sep 07 09:09:46.088936 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:46.088883 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" podStartSLOduration=1.072778725 podStartE2EDuration="2.088862934s" totalImagesPullingTime="1.016084209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:09:46.088259195 +0000 UTC m=+644.605050117" watchObservedRunningTime="2026-09-07 09:09:46.088862934 +0000 UTC m=+644.605653858" Sep 07 09:09:54.614026 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:54.613989 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:54.614446 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:54.614051 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:54.626609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:54.626586 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:09:55.131393 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:09:55.131364 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pndkn" Sep 07 09:10:17.095091 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.095060 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp"] Sep 07 09:10:17.098746 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.098722 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.101611 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.101590 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 07 09:10:17.109802 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.109775 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp"] Sep 07 09:10:17.197924 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.197895 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.198064 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.197934 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.198064 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.197961 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cba918fd-aa93-420a-94af-750652cbe0e0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.198064 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.198032 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvp56\" (UniqueName: \"kubernetes.io/projected/cba918fd-aa93-420a-94af-750652cbe0e0-kube-api-access-zvp56\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.198180 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.198097 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.198180 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.198119 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.298950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.298911 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.298950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.298952 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.299188 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.299006 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.299188 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.299044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.299188 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.299087 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cba918fd-aa93-420a-94af-750652cbe0e0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.299188 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.299114 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvp56\" (UniqueName: \"kubernetes.io/projected/cba918fd-aa93-420a-94af-750652cbe0e0-kube-api-access-zvp56\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.299491 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.299465 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.299582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.299491 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.299582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.299521 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.301169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.301147 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cba918fd-aa93-420a-94af-750652cbe0e0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.301520 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.301502 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cba918fd-aa93-420a-94af-750652cbe0e0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.307340 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.307323 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvp56\" (UniqueName: \"kubernetes.io/projected/cba918fd-aa93-420a-94af-750652cbe0e0-kube-api-access-zvp56\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7rmnp\" (UID: \"cba918fd-aa93-420a-94af-750652cbe0e0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.409390 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.409286 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:17.680375 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:17.680343 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp"] Sep 07 09:10:17.681688 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:10:17.681663 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcba918fd_aa93_420a_94af_750652cbe0e0.slice/crio-0a6ef5d2da661ceb234fe09c69d6a6075f3f73d0ecbd6a50fd618ee718600bc9 WatchSource:0}: Error finding container 0a6ef5d2da661ceb234fe09c69d6a6075f3f73d0ecbd6a50fd618ee718600bc9: Status 404 returned error can't find the container with id 0a6ef5d2da661ceb234fe09c69d6a6075f3f73d0ecbd6a50fd618ee718600bc9 Sep 07 09:10:18.239065 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:18.238982 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" event={"ID":"cba918fd-aa93-420a-94af-750652cbe0e0","Type":"ContainerStarted","Data":"ec4fa3a82c9fe356ba8a8bd8e86131e608562debc715dc98b3282b572e6a2579"} Sep 07 09:10:18.239065 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:18.239030 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" event={"ID":"cba918fd-aa93-420a-94af-750652cbe0e0","Type":"ContainerStarted","Data":"0a6ef5d2da661ceb234fe09c69d6a6075f3f73d0ecbd6a50fd618ee718600bc9"} Sep 07 09:10:18.256515 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:18.256467 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" podStartSLOduration=1.050517539 podStartE2EDuration="1.256451781s" totalImagesPullingTime="205.934242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:10:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:10:18.25455628 +0000 UTC m=+676.771347195" watchObservedRunningTime="2026-09-07 09:10:18.256451781 +0000 UTC m=+676.773242685" Sep 07 09:10:27.409485 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:27.409445 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:27.409896 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:27.409547 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:27.422173 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:27.422145 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:28.306817 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:28.306785 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7rmnp" Sep 07 09:10:42.794332 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.794285 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx"] Sep 07 09:10:42.797606 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.797588 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.799911 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.799892 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 07 09:10:42.807484 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.807458 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx"] Sep 07 09:10:42.818854 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.818829 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.818952 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.818872 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hlhv\" (UniqueName: \"kubernetes.io/projected/45c82f05-4450-4d31-a435-f69f9b96f2b5-kube-api-access-2hlhv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.819017 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.818991 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.819120 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.819101 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c82f05-4450-4d31-a435-f69f9b96f2b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.819170 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.819154 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.819220 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.819205 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.919993 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c82f05-4450-4d31-a435-f69f9b96f2b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920193 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920031 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920193 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920053 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920193 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920387 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920202 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hlhv\" (UniqueName: \"kubernetes.io/projected/45c82f05-4450-4d31-a435-f69f9b96f2b5-kube-api-access-2hlhv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920387 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920256 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920387 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920371 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920445 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.920554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.920538 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.922275 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.922255 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45c82f05-4450-4d31-a435-f69f9b96f2b5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.922587 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.922570 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45c82f05-4450-4d31-a435-f69f9b96f2b5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:42.928060 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:42.928038 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hlhv\" (UniqueName: \"kubernetes.io/projected/45c82f05-4450-4d31-a435-f69f9b96f2b5-kube-api-access-2hlhv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-dg7kx\" (UID: \"45c82f05-4450-4d31-a435-f69f9b96f2b5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:43.109966 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:43.109875 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:43.382268 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:43.382242 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx"] Sep 07 09:10:43.383833 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:10:43.383810 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c82f05_4450_4d31_a435_f69f9b96f2b5.slice/crio-180cbe47669f6ef4a524d9c8a76ad1ea72a7128d2f6a73424c9c585178135c7d WatchSource:0}: Error finding container 180cbe47669f6ef4a524d9c8a76ad1ea72a7128d2f6a73424c9c585178135c7d: Status 404 returned error can't find the container with id 180cbe47669f6ef4a524d9c8a76ad1ea72a7128d2f6a73424c9c585178135c7d Sep 07 09:10:44.381589 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:44.381547 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" event={"ID":"45c82f05-4450-4d31-a435-f69f9b96f2b5","Type":"ContainerStarted","Data":"c5026568903aba40dc34ede037bfde9b8d4f361a1da36a9bb64d0f94df3133a4"} Sep 07 09:10:44.381960 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:44.381615 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" event={"ID":"45c82f05-4450-4d31-a435-f69f9b96f2b5","Type":"ContainerStarted","Data":"180cbe47669f6ef4a524d9c8a76ad1ea72a7128d2f6a73424c9c585178135c7d"} Sep 07 09:10:44.402842 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:44.402790 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" podStartSLOduration=2.219260379 podStartE2EDuration="2.402772704s" totalImagesPullingTime="183.512325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:10:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:10:44.400045494 +0000 UTC m=+702.916836394" watchObservedRunningTime="2026-09-07 09:10:44.402772704 +0000 UTC m=+702.919563607" Sep 07 09:10:53.110933 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:53.110893 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:53.111574 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:53.110964 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:53.123326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:53.123286 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:10:53.440195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:10:53.440116 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-dg7kx" Sep 07 09:12:55.158918 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.158883 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4"] Sep 07 09:12:55.162363 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.162343 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.166463 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.166445 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-614a02-dockercfg-l2hv5\"" Sep 07 09:12:55.166688 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.166673 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-614a02\"" Sep 07 09:12:55.178440 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.178417 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4"] Sep 07 09:12:55.207976 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.207954 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbckg\" (UniqueName: \"kubernetes.io/projected/fab3898d-a417-41c7-aa7b-ee3ad84294e1-kube-api-access-bbckg\") pod \"payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.208099 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.208079 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab3898d-a417-41c7-aa7b-ee3ad84294e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.309139 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.309092 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab3898d-a417-41c7-aa7b-ee3ad84294e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.309139 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.309150 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbckg\" (UniqueName: \"kubernetes.io/projected/fab3898d-a417-41c7-aa7b-ee3ad84294e1-kube-api-access-bbckg\") pod \"payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.309963 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.309941 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab3898d-a417-41c7-aa7b-ee3ad84294e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.317256 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.317237 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbckg\" (UniqueName: \"kubernetes.io/projected/fab3898d-a417-41c7-aa7b-ee3ad84294e1-kube-api-access-bbckg\") pod \"payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.472071 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.471987 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:55.747380 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:55.747353 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4"] Sep 07 09:12:55.749113 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:12:55.749086 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab3898d_a417_41c7_aa7b_ee3ad84294e1.slice/crio-2a21cc885929c1b85fa12996ea10843f9d434e5c1c5fc5e5f1c2e124bdbc86e8 WatchSource:0}: Error finding container 2a21cc885929c1b85fa12996ea10843f9d434e5c1c5fc5e5f1c2e124bdbc86e8: Status 404 returned error can't find the container with id 2a21cc885929c1b85fa12996ea10843f9d434e5c1c5fc5e5f1c2e124bdbc86e8 Sep 07 09:12:56.061761 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.061729 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" event={"ID":"fab3898d-a417-41c7-aa7b-ee3ad84294e1","Type":"ContainerStarted","Data":"b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb"} Sep 07 09:12:56.061923 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.061780 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" event={"ID":"fab3898d-a417-41c7-aa7b-ee3ad84294e1","Type":"ContainerStarted","Data":"2a21cc885929c1b85fa12996ea10843f9d434e5c1c5fc5e5f1c2e124bdbc86e8"} Sep 07 09:12:56.061923 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.061856 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:12:56.083131 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.083076 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" podStartSLOduration=1.083060446 podStartE2EDuration="1.083060446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:56.080627432 +0000 UTC m=+834.597418336" watchObservedRunningTime="2026-09-07 09:12:56.083060446 +0000 UTC m=+834.599851348" Sep 07 09:12:56.327706 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.327620 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt"] Sep 07 09:12:56.332588 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.332560 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.335160 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.335134 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-140be7ab-openshift-default-dockercfg-xt94v\"" Sep 07 09:12:56.345083 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.345060 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt"] Sep 07 09:12:56.420490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-token\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420678 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-podinfo\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420678 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420533 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-envoy\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420678 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420572 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlbq9\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-kube-api-access-hlbq9\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420678 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-socket\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420678 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420666 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-credential-socket\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420976 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420697 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-certs\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420976 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420766 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istiod-ca-cert\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.420976 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.420809 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-data\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.521689 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521653 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-token\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.521864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521710 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-podinfo\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.521864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521745 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-envoy\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.521864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521771 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlbq9\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-kube-api-access-hlbq9\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.521864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521805 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-socket\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.521864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521846 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-credential-socket\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.522169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521892 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-certs\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.522169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521954 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istiod-ca-cert\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.522169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.521997 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-data\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.522519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.522483 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-socket\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.522648 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.522577 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-data\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.522866 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.522841 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istiod-ca-cert\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.522866 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.522857 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-certs\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.523009 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.522926 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-credential-socket\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.524471 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.524441 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-podinfo\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.525173 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.525153 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-envoy\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.528932 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.528911 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-token\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.529102 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.529085 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlbq9\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-kube-api-access-hlbq9\") pod \"e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.648543 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.648458 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:56.932695 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.932664 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt"] Sep 07 09:12:56.934243 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:12:56.934214 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd8d471_e7b0_40e3_8818_96fc6595cc7e.slice/crio-cbe2da5dd42048877236ec998f3ba532360a2ed8860683597685bb31a3c02216 WatchSource:0}: Error finding container cbe2da5dd42048877236ec998f3ba532360a2ed8860683597685bb31a3c02216: Status 404 returned error can't find the container with id cbe2da5dd42048877236ec998f3ba532360a2ed8860683597685bb31a3c02216 Sep 07 09:12:56.937489 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.937450 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:12:56.937606 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.937537 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:12:56.937606 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:56.937581 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:12:57.074709 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.074670 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" event={"ID":"1fd8d471-e7b0-40e3-8818-96fc6595cc7e","Type":"ContainerStarted","Data":"c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839"} Sep 07 09:12:57.074895 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.074730 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" event={"ID":"1fd8d471-e7b0-40e3-8818-96fc6595cc7e","Type":"ContainerStarted","Data":"cbe2da5dd42048877236ec998f3ba532360a2ed8860683597685bb31a3c02216"} Sep 07 09:12:57.094990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.094942 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" podStartSLOduration=1.094927696 podStartE2EDuration="1.094927696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:57.092086884 +0000 UTC m=+835.608877787" watchObservedRunningTime="2026-09-07 09:12:57.094927696 +0000 UTC m=+835.611718599" Sep 07 09:12:57.205673 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.205584 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj"] Sep 07 09:12:57.209417 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.209393 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.212184 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.212159 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-da4d4c-openshift-default-dockercfg-54hdg\"" Sep 07 09:12:57.221938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.221913 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj"] Sep 07 09:12:57.230566 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.230539 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-podinfo\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.230705 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.230597 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d74f0c91-14c6-4740-a2da-f885ef8e287c-istiod-ca-cert\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.230705 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.230669 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-credential-socket\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.230826 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.230765 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-token\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.230826 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.230801 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-certs\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.230938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.230872 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-data\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.230999 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.230982 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt4zz\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-kube-api-access-bt4zz\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.231084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.231067 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-socket\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.231151 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.231106 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-envoy\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332253 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-data\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332373 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt4zz\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-kube-api-access-bt4zz\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332443 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-socket\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332475 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-envoy\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332511 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-podinfo\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332551 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d74f0c91-14c6-4740-a2da-f885ef8e287c-istiod-ca-cert\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332578 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-credential-socket\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332622 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-token\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.332730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.332650 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-certs\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.333105 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.333083 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-certs\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.333402 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.333382 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-data\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.333956 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.333930 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-socket\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.334982 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.334960 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d74f0c91-14c6-4740-a2da-f885ef8e287c-istiod-ca-cert\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.336093 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.336071 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-credential-socket\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.336590 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.336569 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-envoy\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.337981 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.337958 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-podinfo\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.341146 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.341125 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt4zz\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-kube-api-access-bt4zz\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.342635 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.342617 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-token\") pod \"e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.522481 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.522446 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:57.649228 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.649198 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:57.798794 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.798765 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj"] Sep 07 09:12:57.801368 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:12:57.801337 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74f0c91_14c6_4740_a2da_f885ef8e287c.slice/crio-1764929b0b2d94a15cf7bf800451ebd09d342ae912a28cf99ec965764ef4cca6 WatchSource:0}: Error finding container 1764929b0b2d94a15cf7bf800451ebd09d342ae912a28cf99ec965764ef4cca6: Status 404 returned error can't find the container with id 1764929b0b2d94a15cf7bf800451ebd09d342ae912a28cf99ec965764ef4cca6 Sep 07 09:12:57.803542 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.803508 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:12:57.803638 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.803592 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:12:57.803638 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:57.803619 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:12:58.081158 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:58.081071 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" event={"ID":"d74f0c91-14c6-4740-a2da-f885ef8e287c","Type":"ContainerStarted","Data":"d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e"} Sep 07 09:12:58.081158 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:58.081137 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" event={"ID":"d74f0c91-14c6-4740-a2da-f885ef8e287c","Type":"ContainerStarted","Data":"1764929b0b2d94a15cf7bf800451ebd09d342ae912a28cf99ec965764ef4cca6"} Sep 07 09:12:58.118883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:58.118834 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podStartSLOduration=1.118816026 podStartE2EDuration="1.118816026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:12:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:12:58.116289853 +0000 UTC m=+836.633080756" watchObservedRunningTime="2026-09-07 09:12:58.118816026 +0000 UTC m=+836.635606947" Sep 07 09:12:58.523391 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:58.523354 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:58.529117 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:58.529094 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:58.653961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:58.653931 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:59.087171 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:59.087140 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:59.087407 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:59.087387 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:12:59.088104 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:59.088082 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:12:59.088190 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:12:59.088137 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:13:01.801198 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.801163 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w"] Sep 07 09:13:01.804698 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.804682 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:01.807280 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.807257 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-da4d4c-dockercfg-z4bsq\"" Sep 07 09:13:01.807280 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.807274 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-da4d4c\"" Sep 07 09:13:01.812976 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.812955 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w"] Sep 07 09:13:01.882145 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.882110 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205877fb-a353-4c62-9799-5132a7e8c203-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:01.882145 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.882144 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh22v\" (UniqueName: \"kubernetes.io/projected/205877fb-a353-4c62-9799-5132a7e8c203-kube-api-access-xh22v\") pod \"payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:01.983560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.983511 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh22v\" (UniqueName: \"kubernetes.io/projected/205877fb-a353-4c62-9799-5132a7e8c203-kube-api-access-xh22v\") pod \"payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:01.983792 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.983773 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205877fb-a353-4c62-9799-5132a7e8c203-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:01.986137 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.986121 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-da4d4c\"" Sep 07 09:13:01.991619 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.991592 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh22v\" (UniqueName: \"kubernetes.io/projected/205877fb-a353-4c62-9799-5132a7e8c203-kube-api-access-xh22v\") pod \"payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:01.995080 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:01.995056 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205877fb-a353-4c62-9799-5132a7e8c203-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:02.118107 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:02.118009 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-da4d4c-dockercfg-z4bsq\"" Sep 07 09:13:02.125842 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:02.125819 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:02.404249 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:02.404220 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w"] Sep 07 09:13:02.405008 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:13:02.404979 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205877fb_a353_4c62_9799_5132a7e8c203.slice/crio-0efadc0f32b1e1af5252295c200a4be718110fe17b6d0a25e3025d72ba3408be WatchSource:0}: Error finding container 0efadc0f32b1e1af5252295c200a4be718110fe17b6d0a25e3025d72ba3408be: Status 404 returned error can't find the container with id 0efadc0f32b1e1af5252295c200a4be718110fe17b6d0a25e3025d72ba3408be Sep 07 09:13:03.112134 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:03.112093 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" event={"ID":"205877fb-a353-4c62-9799-5132a7e8c203","Type":"ContainerStarted","Data":"e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102"} Sep 07 09:13:03.112624 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:03.112186 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" event={"ID":"205877fb-a353-4c62-9799-5132a7e8c203","Type":"ContainerStarted","Data":"0efadc0f32b1e1af5252295c200a4be718110fe17b6d0a25e3025d72ba3408be"} Sep 07 09:13:03.112624 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:03.112221 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:03.131435 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:03.131358 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" podStartSLOduration=2.131343753 podStartE2EDuration="2.131343753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:03.130912627 +0000 UTC m=+841.647703551" watchObservedRunningTime="2026-09-07 09:13:03.131343753 +0000 UTC m=+841.648134677" Sep 07 09:13:07.075864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:07.075831 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:13:14.123544 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:14.123499 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:13:15.129522 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:15.129475 2558 patch_prober.go:28] interesting pod/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 07 09:13:15.129988 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:15.129547 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 09:13:17.212463 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:17.212431 2558 generic.go:356] "Generic (PLEG): container finished" podID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerID="d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e" exitCode=0 Sep 07 09:13:17.212874 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:17.212513 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" event={"ID":"d74f0c91-14c6-4740-a2da-f885ef8e287c","Type":"ContainerDied","Data":"d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e"} Sep 07 09:13:17.212953 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:17.212939 2558 scope.go:117] "RemoveContainer" containerID="d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e" Sep 07 09:13:17.214110 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:17.214081 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:17.214223 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:17.214148 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:17.214223 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:17.214185 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:18.222716 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.222670 2558 generic.go:356] "Generic (PLEG): container finished" podID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerID="c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839" exitCode=0 Sep 07 09:13:18.223238 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.222807 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" event={"ID":"1fd8d471-e7b0-40e3-8818-96fc6595cc7e","Type":"ContainerDied","Data":"c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839"} Sep 07 09:13:18.223346 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.223314 2558 scope.go:117] "RemoveContainer" containerID="c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839" Sep 07 09:13:18.224917 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.224874 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:18.225013 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.224974 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:18.225087 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.225021 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:18.229399 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.229371 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" event={"ID":"d74f0c91-14c6-4740-a2da-f885ef8e287c","Type":"ContainerStarted","Data":"4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac"} Sep 07 09:13:18.522902 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.522863 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:13:18.523280 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.523251 2558 patch_prober.go:28] interesting pod/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 07 09:13:18.523431 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.523325 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 07 09:13:18.774218 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.774103 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84"] Sep 07 09:13:18.779086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.779060 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.781595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.781573 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-3bcf58-openshift-default-dockercfg-hjfn7\"" Sep 07 09:13:18.789231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.789209 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84"] Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.864822 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-socket\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.864892 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l92nk\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-kube-api-access-l92nk\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.864928 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-data\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.864961 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-token\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.864989 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-envoy\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.865026 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-certs\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.865070 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-credential-socket\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.865092 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-podinfo\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.865247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.865130 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istiod-ca-cert\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.966701 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966650 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-certs\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.966907 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966713 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-credential-socket\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.966907 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966737 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-podinfo\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.966907 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966774 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istiod-ca-cert\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.966907 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966846 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-socket\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.966907 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966889 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l92nk\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-kube-api-access-l92nk\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.967172 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966921 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-data\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.967172 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966956 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-token\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.967172 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.966988 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-envoy\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.967921 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.967770 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-credential-socket\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.968155 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.968127 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istiod-ca-cert\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.968384 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.968336 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-certs\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.968656 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.968635 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-socket\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.968796 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.968772 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-data\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.970155 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.970133 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-envoy\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.970491 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.970476 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-podinfo\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.975410 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.975384 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-token\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:18.975619 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:18.975603 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l92nk\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-kube-api-access-l92nk\") pod \"e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:19.092265 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.092192 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:19.278331 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.278205 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" event={"ID":"1fd8d471-e7b0-40e3-8818-96fc6595cc7e","Type":"ContainerStarted","Data":"543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220"} Sep 07 09:13:19.423556 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.423167 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84"] Sep 07 09:13:19.432969 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:13:19.432922 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c935805_cca1_42d0_a0d4_f10b6e2c6cc8.slice/crio-1993d8a11ab57c14c1614a6dfe8d8181ca4e52ffc273617f9aa97f5dbbb2ae64 WatchSource:0}: Error finding container 1993d8a11ab57c14c1614a6dfe8d8181ca4e52ffc273617f9aa97f5dbbb2ae64: Status 404 returned error can't find the container with id 1993d8a11ab57c14c1614a6dfe8d8181ca4e52ffc273617f9aa97f5dbbb2ae64 Sep 07 09:13:19.439421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.439375 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:19.439540 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.439471 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:19.439540 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.439516 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:19.523026 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.522947 2558 patch_prober.go:28] interesting pod/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 07 09:13:19.523192 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.523024 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 07 09:13:19.649793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.649040 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:13:19.649793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.649479 2558 patch_prober.go:28] interesting pod/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 07 09:13:19.649793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:19.649532 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 07 09:13:20.293012 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:20.292970 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" event={"ID":"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8","Type":"ContainerStarted","Data":"7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b"} Sep 07 09:13:20.293611 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:20.293586 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" event={"ID":"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8","Type":"ContainerStarted","Data":"1993d8a11ab57c14c1614a6dfe8d8181ca4e52ffc273617f9aa97f5dbbb2ae64"} Sep 07 09:13:20.314133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:20.314087 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podStartSLOduration=2.314072685 podStartE2EDuration="2.314072685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:20.312208721 +0000 UTC m=+858.828999625" watchObservedRunningTime="2026-09-07 09:13:20.314072685 +0000 UTC m=+858.830863588" Sep 07 09:13:20.523673 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:20.523631 2558 patch_prober.go:28] interesting pod/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 07 09:13:20.523976 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:20.523941 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 07 09:13:20.650018 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:20.649912 2558 patch_prober.go:28] interesting pod/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 07 09:13:20.650318 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:20.650254 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 07 09:13:21.092958 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.092916 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:21.098446 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.098421 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:21.299224 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.299193 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:21.300068 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.300049 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:21.523739 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.523698 2558 patch_prober.go:28] interesting pod/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 07 09:13:21.523960 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.523775 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 07 09:13:21.649823 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.649781 2558 patch_prober.go:28] interesting pod/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 07 09:13:21.650022 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:21.649848 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 07 09:13:22.523250 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:22.523216 2558 patch_prober.go:28] interesting pod/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 07 09:13:22.523809 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:22.523778 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 07 09:13:22.649927 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:22.649892 2558 patch_prober.go:28] interesting pod/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 07 09:13:22.650233 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:22.650190 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 07 09:13:23.522969 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.522926 2558 patch_prober.go:28] interesting pod/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Sep 07 09:13:23.523192 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.522999 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Sep 07 09:13:23.648933 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.648898 2558 patch_prober.go:28] interesting pod/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Sep 07 09:13:23.649485 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.649452 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Sep 07 09:13:23.942812 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.942711 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m"] Sep 07 09:13:23.947231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.947201 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" Sep 07 09:13:23.949482 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.949452 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:13:23.949684 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.949669 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:13:23.955547 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:23.955523 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m"] Sep 07 09:13:24.036888 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.036844 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01069a90-f9cc-43e4-abac-530d874e6874-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m\" (UID: \"01069a90-f9cc-43e4-abac-530d874e6874\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" Sep 07 09:13:24.141208 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.139970 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01069a90-f9cc-43e4-abac-530d874e6874-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m\" (UID: \"01069a90-f9cc-43e4-abac-530d874e6874\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" Sep 07 09:13:24.142053 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.142027 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01069a90-f9cc-43e4-abac-530d874e6874-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m\" (UID: \"01069a90-f9cc-43e4-abac-530d874e6874\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" Sep 07 09:13:24.255202 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.255113 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk"] Sep 07 09:13:24.258997 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.258971 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.261595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.261571 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-3bcf58\"" Sep 07 09:13:24.261716 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.261662 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-3bcf58-dockercfg-2sm9c\"" Sep 07 09:13:24.261855 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.261826 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" Sep 07 09:13:24.270084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.270039 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk"] Sep 07 09:13:24.443575 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.443540 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dab712ab-0543-4057-9c07-dc945c7b454e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.443575 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.443571 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ggnk\" (UniqueName: \"kubernetes.io/projected/dab712ab-0543-4057-9c07-dc945c7b454e-kube-api-access-7ggnk\") pod \"payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.527265 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.527233 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:13:24.527548 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.527522 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:13:24.528275 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.528257 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:13:24.544865 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.544819 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dab712ab-0543-4057-9c07-dc945c7b454e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.545130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.544876 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ggnk\" (UniqueName: \"kubernetes.io/projected/dab712ab-0543-4057-9c07-dc945c7b454e-kube-api-access-7ggnk\") pod \"payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.546791 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.546738 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dab712ab-0543-4057-9c07-dc945c7b454e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.554582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.554557 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m"] Sep 07 09:13:24.555838 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.555806 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ggnk\" (UniqueName: \"kubernetes.io/projected/dab712ab-0543-4057-9c07-dc945c7b454e-kube-api-access-7ggnk\") pod \"payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.556874 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:13:24.556848 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01069a90_f9cc_43e4_abac_530d874e6874.slice/crio-0e1a54b117b9fff14ec2147137b01f6519dd8b9a818a9d6e590d9172279f2a73 WatchSource:0}: Error finding container 0e1a54b117b9fff14ec2147137b01f6519dd8b9a818a9d6e590d9172279f2a73: Status 404 returned error can't find the container with id 0e1a54b117b9fff14ec2147137b01f6519dd8b9a818a9d6e590d9172279f2a73 Sep 07 09:13:24.573166 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.572966 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:24.719036 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.718980 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:13:24.719521 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.719436 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:13:24.720749 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.720724 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:13:24.857479 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:24.857457 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk"] Sep 07 09:13:24.859149 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:13:24.859126 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab712ab_0543_4057_9c07_dc945c7b454e.slice/crio-7ea2638701dcb6e35115e84c890226ad901f86d2440aeaa2fccb4b4c00322485 WatchSource:0}: Error finding container 7ea2638701dcb6e35115e84c890226ad901f86d2440aeaa2fccb4b4c00322485: Status 404 returned error can't find the container with id 7ea2638701dcb6e35115e84c890226ad901f86d2440aeaa2fccb4b4c00322485 Sep 07 09:13:25.339913 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:25.339874 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" event={"ID":"dab712ab-0543-4057-9c07-dc945c7b454e","Type":"ContainerStarted","Data":"8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124"} Sep 07 09:13:25.340115 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:25.339938 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" event={"ID":"dab712ab-0543-4057-9c07-dc945c7b454e","Type":"ContainerStarted","Data":"7ea2638701dcb6e35115e84c890226ad901f86d2440aeaa2fccb4b4c00322485"} Sep 07 09:13:25.340115 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:25.339980 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:25.341353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:25.341326 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" event={"ID":"01069a90-f9cc-43e4-abac-530d874e6874","Type":"ContainerStarted","Data":"0e1a54b117b9fff14ec2147137b01f6519dd8b9a818a9d6e590d9172279f2a73"} Sep 07 09:13:25.357482 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:25.357440 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" podStartSLOduration=1.357424808 podStartE2EDuration="1.357424808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:25.355079164 +0000 UTC m=+863.871870069" watchObservedRunningTime="2026-09-07 09:13:25.357424808 +0000 UTC m=+863.874215710" Sep 07 09:13:28.364938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:28.364895 2558 generic.go:356] "Generic (PLEG): container finished" podID="01069a90-f9cc-43e4-abac-530d874e6874" containerID="1e25e1bf72820c75edfcacaae8e6f0918791ece1d928a65a48517debf502a345" exitCode=0 Sep 07 09:13:28.365332 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:28.364952 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" event={"ID":"01069a90-f9cc-43e4-abac-530d874e6874","Type":"ContainerDied","Data":"1e25e1bf72820c75edfcacaae8e6f0918791ece1d928a65a48517debf502a345"} Sep 07 09:13:29.548219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:29.548188 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" Sep 07 09:13:29.598366 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:29.598329 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01069a90-f9cc-43e4-abac-530d874e6874-maas-api-service-ca\") pod \"01069a90-f9cc-43e4-abac-530d874e6874\" (UID: \"01069a90-f9cc-43e4-abac-530d874e6874\") " Sep 07 09:13:29.598614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:29.598589 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01069a90-f9cc-43e4-abac-530d874e6874-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "01069a90-f9cc-43e4-abac-530d874e6874" (UID: "01069a90-f9cc-43e4-abac-530d874e6874"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:13:29.699144 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:29.699061 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/01069a90-f9cc-43e4-abac-530d874e6874-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:30.378557 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:30.378526 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" Sep 07 09:13:30.378733 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:30.378561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m" event={"ID":"01069a90-f9cc-43e4-abac-530d874e6874","Type":"ContainerDied","Data":"0e1a54b117b9fff14ec2147137b01f6519dd8b9a818a9d6e590d9172279f2a73"} Sep 07 09:13:30.378733 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:30.378609 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e1a54b117b9fff14ec2147137b01f6519dd8b9a818a9d6e590d9172279f2a73" Sep 07 09:13:36.350240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:36.350208 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:13:37.339487 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:37.339441 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 07 09:13:37.339703 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:37.339519 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 09:13:38.442674 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:38.442637 2558 generic.go:356] "Generic (PLEG): container finished" podID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerID="7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b" exitCode=0 Sep 07 09:13:38.443083 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:38.442711 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" event={"ID":"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8","Type":"ContainerDied","Data":"7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b"} Sep 07 09:13:38.443128 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:38.443113 2558 scope.go:117] "RemoveContainer" containerID="7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b" Sep 07 09:13:38.444201 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:38.444174 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:38.444275 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:38.444245 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:38.444336 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:38.444324 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:13:39.450686 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:39.450649 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" event={"ID":"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8","Type":"ContainerStarted","Data":"129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae"} Sep 07 09:13:40.092656 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:40.092615 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:40.092869 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:40.092845 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 07 09:13:40.092930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:40.092909 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 07 09:13:40.714206 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:40.714166 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m"] Sep 07 09:13:40.717554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:40.717521 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-140be7ab-j2t2m"] Sep 07 09:13:40.857820 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:40.857786 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt"] Sep 07 09:13:40.858255 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:40.858228 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" containerID="cri-o://543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220" gracePeriod=30 Sep 07 09:13:41.093291 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:41.093254 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 07 09:13:41.093478 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:41.093336 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 07 09:13:41.955252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:41.955211 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01069a90-f9cc-43e4-abac-530d874e6874" path="/var/lib/kubelet/pods/01069a90-f9cc-43e4-abac-530d874e6874/volumes" Sep 07 09:13:42.093503 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:42.093472 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 07 09:13:42.093671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:42.093523 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 07 09:13:43.092882 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:43.092852 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 07 09:13:43.093247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:43.092901 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 07 09:13:44.093368 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:44.093330 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 07 09:13:44.093849 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:44.093395 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 07 09:13:45.092801 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:45.092769 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 07 09:13:45.092969 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:45.092822 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 07 09:13:46.096372 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.096344 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:46.096784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.096764 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:46.097458 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.097436 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:13:46.133652 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.133631 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:13:46.188987 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.188902 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-podinfo\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.188987 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.188946 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-envoy\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189001 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlbq9\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-kube-api-access-hlbq9\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189057 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-credential-socket\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189105 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-socket\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189137 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-certs\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189165 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-token\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189506 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189263 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-data\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189618 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189594 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istiod-ca-cert\") pod \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\" (UID: \"1fd8d471-e7b0-40e3-8818-96fc6595cc7e\") " Sep 07 09:13:46.189701 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189672 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:13:46.189897 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.189875 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:13:46.190101 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.190084 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.190166 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.190105 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.190343 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.190321 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:13:46.191205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.190371 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:13:46.191205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.190772 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:13:46.191744 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.191700 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:13:46.191818 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.191790 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:13:46.191818 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.191791 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:13:46.193059 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.193036 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-kube-api-access-hlbq9" (OuterVolumeSpecName: "kube-api-access-hlbq9") pod "1fd8d471-e7b0-40e3-8818-96fc6595cc7e" (UID: "1fd8d471-e7b0-40e3-8818-96fc6595cc7e"). InnerVolumeSpecName "kube-api-access-hlbq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:13:46.293595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.291401 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlbq9\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-kube-api-access-hlbq9\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.293595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.291434 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.293595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.291450 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.293595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.291465 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.293595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.291478 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.293595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.291489 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.293595 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.291501 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd8d471-e7b0-40e3-8818-96fc6595cc7e-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:13:46.495793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.495706 2558 generic.go:356] "Generic (PLEG): container finished" podID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerID="543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220" exitCode=0 Sep 07 09:13:46.495793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.495745 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" event={"ID":"1fd8d471-e7b0-40e3-8818-96fc6595cc7e","Type":"ContainerDied","Data":"543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220"} Sep 07 09:13:46.495793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.495763 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" Sep 07 09:13:46.496074 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.495811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt" event={"ID":"1fd8d471-e7b0-40e3-8818-96fc6595cc7e","Type":"ContainerDied","Data":"cbe2da5dd42048877236ec998f3ba532360a2ed8860683597685bb31a3c02216"} Sep 07 09:13:46.496074 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.495839 2558 scope.go:117] "RemoveContainer" containerID="543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220" Sep 07 09:13:46.505532 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.505510 2558 scope.go:117] "RemoveContainer" containerID="c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839" Sep 07 09:13:46.513704 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.513686 2558 scope.go:117] "RemoveContainer" containerID="543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220" Sep 07 09:13:46.513959 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:13:46.513939 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220\": container with ID starting with 543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220 not found: ID does not exist" containerID="543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220" Sep 07 09:13:46.514003 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.513969 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220"} err="failed to get container status \"543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220\": rpc error: code = NotFound desc = could not find container \"543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220\": container with ID starting with 543c6af092245b84940526bf0f0f93e05e7b4b9d60ecc6cdf45eee8acbcee220 not found: ID does not exist" Sep 07 09:13:46.514003 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.513986 2558 scope.go:117] "RemoveContainer" containerID="c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839" Sep 07 09:13:46.514240 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:13:46.514215 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839\": container with ID starting with c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839 not found: ID does not exist" containerID="c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839" Sep 07 09:13:46.514281 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.514246 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839"} err="failed to get container status \"c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839\": rpc error: code = NotFound desc = could not find container \"c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839\": container with ID starting with c5dc82c92ca391c639d7aefd0ee2c54b94c2e1f3d4d1c6bd083fcce0714f2839 not found: ID does not exist" Sep 07 09:13:46.520451 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.520429 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt"] Sep 07 09:13:46.523799 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:46.523777 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-140be7ab-openshift-default-5456659c5b-5g5wt"] Sep 07 09:13:47.951855 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:13:47.951821 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" path="/var/lib/kubelet/pods/1fd8d471-e7b0-40e3-8818-96fc6595cc7e/volumes" Sep 07 09:14:01.939288 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:01.939214 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:14:01.939858 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:01.939750 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:14:08.932843 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.932804 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks"] Sep 07 09:14:08.933240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933193 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" Sep 07 09:14:08.933240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933204 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" Sep 07 09:14:08.933240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933217 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" Sep 07 09:14:08.933240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933223 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" Sep 07 09:14:08.933240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933232 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01069a90-f9cc-43e4-abac-530d874e6874" containerName="revoke-keys" Sep 07 09:14:08.933240 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933239 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="01069a90-f9cc-43e4-abac-530d874e6874" containerName="revoke-keys" Sep 07 09:14:08.933488 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933326 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="01069a90-f9cc-43e4-abac-530d874e6874" containerName="revoke-keys" Sep 07 09:14:08.933488 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.933338 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" Sep 07 09:14:08.937646 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.937629 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" Sep 07 09:14:08.940433 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.940411 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:14:08.940433 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.940422 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:14:08.943574 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.943553 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks"] Sep 07 09:14:08.995852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:08.995815 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c73a6eb7-163f-4e1a-8565-88a2c7ccfdba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7928b616-cgdks\" (UID: \"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" Sep 07 09:14:09.096378 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:09.096339 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c73a6eb7-163f-4e1a-8565-88a2c7ccfdba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7928b616-cgdks\" (UID: \"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" Sep 07 09:14:09.096937 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:09.096916 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c73a6eb7-163f-4e1a-8565-88a2c7ccfdba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7928b616-cgdks\" (UID: \"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" Sep 07 09:14:09.249455 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:09.249375 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" Sep 07 09:14:09.518257 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:09.518226 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks"] Sep 07 09:14:09.519099 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:14:09.519069 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73a6eb7_163f_4e1a_8565_88a2c7ccfdba.slice/crio-79a3cc11151f6fe01b40e584b8b499a247720a4ab454fe684df337d0451adf94 WatchSource:0}: Error finding container 79a3cc11151f6fe01b40e584b8b499a247720a4ab454fe684df337d0451adf94: Status 404 returned error can't find the container with id 79a3cc11151f6fe01b40e584b8b499a247720a4ab454fe684df337d0451adf94 Sep 07 09:14:09.636341 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:09.636287 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" event={"ID":"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba","Type":"ContainerStarted","Data":"79a3cc11151f6fe01b40e584b8b499a247720a4ab454fe684df337d0451adf94"} Sep 07 09:14:10.643723 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:10.643684 2558 generic.go:356] "Generic (PLEG): container finished" podID="c73a6eb7-163f-4e1a-8565-88a2c7ccfdba" containerID="32a53f3a685747e28fc91d7bae5a4d1c5777d83c7d6976e9d7abd5adf5d48707" exitCode=0 Sep 07 09:14:10.644112 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:10.643768 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" event={"ID":"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba","Type":"ContainerDied","Data":"32a53f3a685747e28fc91d7bae5a4d1c5777d83c7d6976e9d7abd5adf5d48707"} Sep 07 09:14:11.783818 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:11.783795 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" Sep 07 09:14:11.823773 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:11.823738 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c73a6eb7-163f-4e1a-8565-88a2c7ccfdba-maas-api-service-ca\") pod \"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba\" (UID: \"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba\") " Sep 07 09:14:11.824118 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:11.824095 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c73a6eb7-163f-4e1a-8565-88a2c7ccfdba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c73a6eb7-163f-4e1a-8565-88a2c7ccfdba" (UID: "c73a6eb7-163f-4e1a-8565-88a2c7ccfdba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:14:11.925023 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:11.924950 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c73a6eb7-163f-4e1a-8565-88a2c7ccfdba-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:14:12.656278 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:12.656242 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" event={"ID":"c73a6eb7-163f-4e1a-8565-88a2c7ccfdba","Type":"ContainerDied","Data":"79a3cc11151f6fe01b40e584b8b499a247720a4ab454fe684df337d0451adf94"} Sep 07 09:14:12.656440 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:12.656319 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79a3cc11151f6fe01b40e584b8b499a247720a4ab454fe684df337d0451adf94" Sep 07 09:14:12.656440 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:12.656260 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks" Sep 07 09:14:15.789734 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:15.789701 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks"] Sep 07 09:14:15.792272 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:15.792244 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7928b616-cgdks"] Sep 07 09:14:15.953363 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:15.953327 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c73a6eb7-163f-4e1a-8565-88a2c7ccfdba" path="/var/lib/kubelet/pods/c73a6eb7-163f-4e1a-8565-88a2c7ccfdba/volumes" Sep 07 09:14:44.901327 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.901282 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f"] Sep 07 09:14:44.901752 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.901692 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c73a6eb7-163f-4e1a-8565-88a2c7ccfdba" containerName="revoke-keys" Sep 07 09:14:44.901752 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.901704 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c73a6eb7-163f-4e1a-8565-88a2c7ccfdba" containerName="revoke-keys" Sep 07 09:14:44.901833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.901766 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c73a6eb7-163f-4e1a-8565-88a2c7ccfdba" containerName="revoke-keys" Sep 07 09:14:44.901833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.901777 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fd8d471-e7b0-40e3-8818-96fc6595cc7e" containerName="istio-proxy" Sep 07 09:14:44.905048 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.905032 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" Sep 07 09:14:44.907339 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.907292 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:14:44.907473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.907429 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:14:44.912441 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:44.912414 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f"] Sep 07 09:14:45.054337 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.054276 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548c2cb9-91f2-4a4e-8228-79e66af2d444-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f\" (UID: \"548c2cb9-91f2-4a4e-8228-79e66af2d444\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" Sep 07 09:14:45.155206 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.155116 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548c2cb9-91f2-4a4e-8228-79e66af2d444-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f\" (UID: \"548c2cb9-91f2-4a4e-8228-79e66af2d444\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" Sep 07 09:14:45.155806 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.155776 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548c2cb9-91f2-4a4e-8228-79e66af2d444-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f\" (UID: \"548c2cb9-91f2-4a4e-8228-79e66af2d444\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" Sep 07 09:14:45.217544 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.217517 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" Sep 07 09:14:45.490771 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.490747 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f"] Sep 07 09:14:45.492266 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:14:45.492238 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod548c2cb9_91f2_4a4e_8228_79e66af2d444.slice/crio-ea0ef8093a6474b22ba66695492bbbf0ac983bcce2bfac02dcfdd8b3ff834b4f WatchSource:0}: Error finding container ea0ef8093a6474b22ba66695492bbbf0ac983bcce2bfac02dcfdd8b3ff834b4f: Status 404 returned error can't find the container with id ea0ef8093a6474b22ba66695492bbbf0ac983bcce2bfac02dcfdd8b3ff834b4f Sep 07 09:14:45.862841 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.862811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" event={"ID":"548c2cb9-91f2-4a4e-8228-79e66af2d444","Type":"ContainerStarted","Data":"4f46d30303631336c88151f27f8c4c9f7d31cfc885e41f4ea29457825b368b3e"} Sep 07 09:14:45.862957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.862865 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" event={"ID":"548c2cb9-91f2-4a4e-8228-79e66af2d444","Type":"ContainerStarted","Data":"ea0ef8093a6474b22ba66695492bbbf0ac983bcce2bfac02dcfdd8b3ff834b4f"} Sep 07 09:14:45.877403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:45.877363 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" podStartSLOduration=1.87734889 podStartE2EDuration="1.87734889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:45.876070133 +0000 UTC m=+944.392861046" watchObservedRunningTime="2026-09-07 09:14:45.87734889 +0000 UTC m=+944.394139792" Sep 07 09:14:46.869697 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:46.869662 2558 generic.go:356] "Generic (PLEG): container finished" podID="548c2cb9-91f2-4a4e-8228-79e66af2d444" containerID="4f46d30303631336c88151f27f8c4c9f7d31cfc885e41f4ea29457825b368b3e" exitCode=0 Sep 07 09:14:46.870109 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:46.869718 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" event={"ID":"548c2cb9-91f2-4a4e-8228-79e66af2d444","Type":"ContainerDied","Data":"4f46d30303631336c88151f27f8c4c9f7d31cfc885e41f4ea29457825b368b3e"} Sep 07 09:14:48.011118 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:48.011089 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" Sep 07 09:14:48.081339 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:48.081281 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548c2cb9-91f2-4a4e-8228-79e66af2d444-maas-api-service-ca\") pod \"548c2cb9-91f2-4a4e-8228-79e66af2d444\" (UID: \"548c2cb9-91f2-4a4e-8228-79e66af2d444\") " Sep 07 09:14:48.081607 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:48.081586 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/548c2cb9-91f2-4a4e-8228-79e66af2d444-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "548c2cb9-91f2-4a4e-8228-79e66af2d444" (UID: "548c2cb9-91f2-4a4e-8228-79e66af2d444"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:14:48.182534 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:48.182444 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/548c2cb9-91f2-4a4e-8228-79e66af2d444-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:14:48.882308 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:48.882267 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" Sep 07 09:14:48.882473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:48.882293 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f" event={"ID":"548c2cb9-91f2-4a4e-8228-79e66af2d444","Type":"ContainerDied","Data":"ea0ef8093a6474b22ba66695492bbbf0ac983bcce2bfac02dcfdd8b3ff834b4f"} Sep 07 09:14:48.882473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:48.882390 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea0ef8093a6474b22ba66695492bbbf0ac983bcce2bfac02dcfdd8b3ff834b4f" Sep 07 09:14:51.075092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:51.075044 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f"] Sep 07 09:14:51.077603 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:51.077576 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc11d6be-4k68f"] Sep 07 09:14:51.951246 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:51.951212 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="548c2cb9-91f2-4a4e-8228-79e66af2d444" path="/var/lib/kubelet/pods/548c2cb9-91f2-4a4e-8228-79e66af2d444/volumes" Sep 07 09:14:57.163960 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.163926 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f"] Sep 07 09:14:57.164527 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.164505 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="548c2cb9-91f2-4a4e-8228-79e66af2d444" containerName="revoke-keys" Sep 07 09:14:57.164614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.164530 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="548c2cb9-91f2-4a4e-8228-79e66af2d444" containerName="revoke-keys" Sep 07 09:14:57.164669 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.164646 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="548c2cb9-91f2-4a4e-8228-79e66af2d444" containerName="revoke-keys" Sep 07 09:14:57.167950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.167929 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.172276 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.172259 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-863fd8cf-dockercfg-46w7t\"" Sep 07 09:14:57.172753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.172739 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-863fd8cf\"" Sep 07 09:14:57.199576 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.199555 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f"] Sep 07 09:14:57.263666 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.261983 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pjxg\" (UniqueName: \"kubernetes.io/projected/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-kube-api-access-4pjxg\") pod \"payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.263666 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.262161 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-plugins-config-volume\") pod \"payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.363763 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.363717 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pjxg\" (UniqueName: \"kubernetes.io/projected/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-kube-api-access-4pjxg\") pod \"payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.363933 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.363829 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-plugins-config-volume\") pod \"payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.364503 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.364485 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-plugins-config-volume\") pod \"payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.371141 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.371114 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pjxg\" (UniqueName: \"kubernetes.io/projected/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-kube-api-access-4pjxg\") pod \"payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.478092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.478011 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.749339 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.749316 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f"] Sep 07 09:14:57.750761 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:14:57.750737 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf852a5_84ff_4cf9_b3a5_abf05ce94e2e.slice/crio-cbdc57bcbb628546edd8c5ebd4c3257a13fc7e2fc7c258af6368b55763042a04 WatchSource:0}: Error finding container cbdc57bcbb628546edd8c5ebd4c3257a13fc7e2fc7c258af6368b55763042a04: Status 404 returned error can't find the container with id cbdc57bcbb628546edd8c5ebd4c3257a13fc7e2fc7c258af6368b55763042a04 Sep 07 09:14:57.952072 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.952043 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:14:57.952208 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.952090 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" event={"ID":"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e","Type":"ContainerStarted","Data":"c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308"} Sep 07 09:14:57.952208 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.952109 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" event={"ID":"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e","Type":"ContainerStarted","Data":"cbdc57bcbb628546edd8c5ebd4c3257a13fc7e2fc7c258af6368b55763042a04"} Sep 07 09:14:57.962499 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:14:57.962454 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" podStartSLOduration=0.962439433 podStartE2EDuration="962.439433ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:57.961245027 +0000 UTC m=+956.478035974" watchObservedRunningTime="2026-09-07 09:14:57.962439433 +0000 UTC m=+956.479230338" Sep 07 09:15:08.956084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:08.956054 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:15:20.204382 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.204353 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r"] Sep 07 09:15:20.208105 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.208089 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" Sep 07 09:15:20.210998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.210977 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:15:20.211127 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.211005 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:15:20.213883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.213861 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r"] Sep 07 09:15:20.387588 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.387542 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e314b88b-bd23-4604-90be-25b6986053b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r\" (UID: \"e314b88b-bd23-4604-90be-25b6986053b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" Sep 07 09:15:20.488101 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.488019 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e314b88b-bd23-4604-90be-25b6986053b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r\" (UID: \"e314b88b-bd23-4604-90be-25b6986053b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" Sep 07 09:15:20.488685 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.488664 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e314b88b-bd23-4604-90be-25b6986053b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r\" (UID: \"e314b88b-bd23-4604-90be-25b6986053b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" Sep 07 09:15:20.521505 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.521474 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" Sep 07 09:15:20.790076 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:20.790050 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r"] Sep 07 09:15:20.791422 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:15:20.791394 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode314b88b_bd23_4604_90be_25b6986053b8.slice/crio-0e51b8bb04fafdb941510967361c0746bd35696be4e65c88366da32417978f52 WatchSource:0}: Error finding container 0e51b8bb04fafdb941510967361c0746bd35696be4e65c88366da32417978f52: Status 404 returned error can't find the container with id 0e51b8bb04fafdb941510967361c0746bd35696be4e65c88366da32417978f52 Sep 07 09:15:21.082130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:21.082048 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" event={"ID":"e314b88b-bd23-4604-90be-25b6986053b8","Type":"ContainerStarted","Data":"0e51b8bb04fafdb941510967361c0746bd35696be4e65c88366da32417978f52"} Sep 07 09:15:22.088730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:22.088694 2558 generic.go:356] "Generic (PLEG): container finished" podID="e314b88b-bd23-4604-90be-25b6986053b8" containerID="8e6e26f6dfd323018043472b6bb37592ade2d0d832da0e8224782a43e6a4366f" exitCode=0 Sep 07 09:15:22.089130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:22.088779 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" event={"ID":"e314b88b-bd23-4604-90be-25b6986053b8","Type":"ContainerDied","Data":"8e6e26f6dfd323018043472b6bb37592ade2d0d832da0e8224782a43e6a4366f"} Sep 07 09:15:23.230826 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:23.230800 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" Sep 07 09:15:23.319424 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:23.319390 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e314b88b-bd23-4604-90be-25b6986053b8-maas-api-service-ca\") pod \"e314b88b-bd23-4604-90be-25b6986053b8\" (UID: \"e314b88b-bd23-4604-90be-25b6986053b8\") " Sep 07 09:15:23.319722 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:23.319698 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e314b88b-bd23-4604-90be-25b6986053b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e314b88b-bd23-4604-90be-25b6986053b8" (UID: "e314b88b-bd23-4604-90be-25b6986053b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:15:23.420655 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:23.420581 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e314b88b-bd23-4604-90be-25b6986053b8-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:24.102023 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.101994 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" Sep 07 09:15:24.102220 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.102034 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r" event={"ID":"e314b88b-bd23-4604-90be-25b6986053b8","Type":"ContainerDied","Data":"0e51b8bb04fafdb941510967361c0746bd35696be4e65c88366da32417978f52"} Sep 07 09:15:24.102220 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.102082 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e51b8bb04fafdb941510967361c0746bd35696be4e65c88366da32417978f52" Sep 07 09:15:24.353231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.353197 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6"] Sep 07 09:15:24.353875 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.353852 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e314b88b-bd23-4604-90be-25b6986053b8" containerName="revoke-keys" Sep 07 09:15:24.353929 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.353882 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e314b88b-bd23-4604-90be-25b6986053b8" containerName="revoke-keys" Sep 07 09:15:24.354018 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.354007 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e314b88b-bd23-4604-90be-25b6986053b8" containerName="revoke-keys" Sep 07 09:15:24.357655 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.357637 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" Sep 07 09:15:24.360177 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.360157 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:15:24.360500 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.360203 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:15:24.362828 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.362807 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6"] Sep 07 09:15:24.533882 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.533844 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41235ffd-849f-47e4-84f3-4f64d6302c68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6\" (UID: \"41235ffd-849f-47e4-84f3-4f64d6302c68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" Sep 07 09:15:24.635680 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.635597 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41235ffd-849f-47e4-84f3-4f64d6302c68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6\" (UID: \"41235ffd-849f-47e4-84f3-4f64d6302c68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" Sep 07 09:15:24.636237 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.636181 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41235ffd-849f-47e4-84f3-4f64d6302c68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6\" (UID: \"41235ffd-849f-47e4-84f3-4f64d6302c68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" Sep 07 09:15:24.669521 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.669497 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" Sep 07 09:15:24.942481 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:24.942443 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6"] Sep 07 09:15:24.943860 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:15:24.943829 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41235ffd_849f_47e4_84f3_4f64d6302c68.slice/crio-db57139448a709a8e9e0ff84821d68fe4c20a51365bcc7c6c1d77be62cf18b70 WatchSource:0}: Error finding container db57139448a709a8e9e0ff84821d68fe4c20a51365bcc7c6c1d77be62cf18b70: Status 404 returned error can't find the container with id db57139448a709a8e9e0ff84821d68fe4c20a51365bcc7c6c1d77be62cf18b70 Sep 07 09:15:25.109581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:25.109546 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" event={"ID":"41235ffd-849f-47e4-84f3-4f64d6302c68","Type":"ContainerStarted","Data":"db57139448a709a8e9e0ff84821d68fe4c20a51365bcc7c6c1d77be62cf18b70"} Sep 07 09:15:26.117355 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:26.117294 2558 generic.go:356] "Generic (PLEG): container finished" podID="41235ffd-849f-47e4-84f3-4f64d6302c68" containerID="0a08ce4fade2545c5ecf3880582c5b53726beda83d27aba57fab61ee39190942" exitCode=0 Sep 07 09:15:26.117742 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:26.117391 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" event={"ID":"41235ffd-849f-47e4-84f3-4f64d6302c68","Type":"ContainerDied","Data":"0a08ce4fade2545c5ecf3880582c5b53726beda83d27aba57fab61ee39190942"} Sep 07 09:15:27.282975 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.282950 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" Sep 07 09:15:27.292612 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.292588 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f"] Sep 07 09:15:27.293029 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.292996 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" podUID="7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" containerName="payload-processing" containerID="cri-o://c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308" gracePeriod=30 Sep 07 09:15:27.360460 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.360410 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41235ffd-849f-47e4-84f3-4f64d6302c68-maas-api-service-ca\") pod \"41235ffd-849f-47e4-84f3-4f64d6302c68\" (UID: \"41235ffd-849f-47e4-84f3-4f64d6302c68\") " Sep 07 09:15:27.360794 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.360767 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41235ffd-849f-47e4-84f3-4f64d6302c68-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41235ffd-849f-47e4-84f3-4f64d6302c68" (UID: "41235ffd-849f-47e4-84f3-4f64d6302c68"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:15:27.449143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.449109 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r"] Sep 07 09:15:27.452196 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.452168 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-863fd8cf-g8d6r"] Sep 07 09:15:27.461704 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.461679 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41235ffd-849f-47e4-84f3-4f64d6302c68-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:27.575984 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.575949 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:15:27.664650 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.664557 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-plugins-config-volume\") pod \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " Sep 07 09:15:27.664650 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.664632 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pjxg\" (UniqueName: \"kubernetes.io/projected/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-kube-api-access-4pjxg\") pod \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\" (UID: \"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e\") " Sep 07 09:15:27.664957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.664931 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" (UID: "7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:15:27.666661 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.666641 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-kube-api-access-4pjxg" (OuterVolumeSpecName: "kube-api-access-4pjxg") pod "7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" (UID: "7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e"). InnerVolumeSpecName "kube-api-access-4pjxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:15:27.765513 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.765480 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pjxg\" (UniqueName: \"kubernetes.io/projected/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-kube-api-access-4pjxg\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:27.765513 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.765506 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:27.952430 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:27.952349 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e314b88b-bd23-4604-90be-25b6986053b8" path="/var/lib/kubelet/pods/e314b88b-bd23-4604-90be-25b6986053b8/volumes" Sep 07 09:15:28.131068 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.131036 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" Sep 07 09:15:28.131252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.131076 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6" event={"ID":"41235ffd-849f-47e4-84f3-4f64d6302c68","Type":"ContainerDied","Data":"db57139448a709a8e9e0ff84821d68fe4c20a51365bcc7c6c1d77be62cf18b70"} Sep 07 09:15:28.131252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.131129 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db57139448a709a8e9e0ff84821d68fe4c20a51365bcc7c6c1d77be62cf18b70" Sep 07 09:15:28.132571 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.132538 2558 generic.go:356] "Generic (PLEG): container finished" podID="7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" containerID="c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308" exitCode=0 Sep 07 09:15:28.132690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.132573 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" Sep 07 09:15:28.132690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.132595 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" event={"ID":"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e","Type":"ContainerDied","Data":"c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308"} Sep 07 09:15:28.132690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.132639 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f" event={"ID":"7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e","Type":"ContainerDied","Data":"cbdc57bcbb628546edd8c5ebd4c3257a13fc7e2fc7c258af6368b55763042a04"} Sep 07 09:15:28.132690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.132660 2558 scope.go:117] "RemoveContainer" containerID="c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308" Sep 07 09:15:28.142714 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.142697 2558 scope.go:117] "RemoveContainer" containerID="c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308" Sep 07 09:15:28.143003 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:15:28.142985 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308\": container with ID starting with c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308 not found: ID does not exist" containerID="c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308" Sep 07 09:15:28.143072 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.143015 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308"} err="failed to get container status \"c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308\": rpc error: code = NotFound desc = could not find container \"c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308\": container with ID starting with c26b318108789d7a096e9e632e2646f6add6d45bef01aa0fd5a4ea5d13973308 not found: ID does not exist" Sep 07 09:15:28.149608 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.149584 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f"] Sep 07 09:15:28.153267 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:28.153248 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-863fd8cf-5bf759df6f-tc59f"] Sep 07 09:15:29.952852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:29.952821 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" path="/var/lib/kubelet/pods/7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e/volumes" Sep 07 09:15:35.238769 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.238726 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w"] Sep 07 09:15:35.239198 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.238941 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" podUID="205877fb-a353-4c62-9799-5132a7e8c203" containerName="payload-pre-processing" containerID="cri-o://e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102" gracePeriod=30 Sep 07 09:15:35.395929 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.395893 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6"] Sep 07 09:15:35.398201 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.398178 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-da4d4c-qgxh6"] Sep 07 09:15:35.499005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.498945 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:15:35.539369 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.539342 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh22v\" (UniqueName: \"kubernetes.io/projected/205877fb-a353-4c62-9799-5132a7e8c203-kube-api-access-xh22v\") pod \"205877fb-a353-4c62-9799-5132a7e8c203\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " Sep 07 09:15:35.539520 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.539428 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205877fb-a353-4c62-9799-5132a7e8c203-plugins-config-volume\") pod \"205877fb-a353-4c62-9799-5132a7e8c203\" (UID: \"205877fb-a353-4c62-9799-5132a7e8c203\") " Sep 07 09:15:35.539803 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.539775 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/205877fb-a353-4c62-9799-5132a7e8c203-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "205877fb-a353-4c62-9799-5132a7e8c203" (UID: "205877fb-a353-4c62-9799-5132a7e8c203"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:15:35.541500 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.541479 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/205877fb-a353-4c62-9799-5132a7e8c203-kube-api-access-xh22v" (OuterVolumeSpecName: "kube-api-access-xh22v") pod "205877fb-a353-4c62-9799-5132a7e8c203" (UID: "205877fb-a353-4c62-9799-5132a7e8c203"). InnerVolumeSpecName "kube-api-access-xh22v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:15:35.640254 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.640221 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/205877fb-a353-4c62-9799-5132a7e8c203-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:35.640254 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.640247 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh22v\" (UniqueName: \"kubernetes.io/projected/205877fb-a353-4c62-9799-5132a7e8c203-kube-api-access-xh22v\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:35.951618 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:35.951586 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41235ffd-849f-47e4-84f3-4f64d6302c68" path="/var/lib/kubelet/pods/41235ffd-849f-47e4-84f3-4f64d6302c68/volumes" Sep 07 09:15:36.181892 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.181861 2558 generic.go:356] "Generic (PLEG): container finished" podID="205877fb-a353-4c62-9799-5132a7e8c203" containerID="e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102" exitCode=0 Sep 07 09:15:36.182092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.181907 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" Sep 07 09:15:36.182092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.181919 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" event={"ID":"205877fb-a353-4c62-9799-5132a7e8c203","Type":"ContainerDied","Data":"e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102"} Sep 07 09:15:36.182092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.181989 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w" event={"ID":"205877fb-a353-4c62-9799-5132a7e8c203","Type":"ContainerDied","Data":"0efadc0f32b1e1af5252295c200a4be718110fe17b6d0a25e3025d72ba3408be"} Sep 07 09:15:36.182092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.182020 2558 scope.go:117] "RemoveContainer" containerID="e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102" Sep 07 09:15:36.191346 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.191328 2558 scope.go:117] "RemoveContainer" containerID="e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102" Sep 07 09:15:36.191624 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:15:36.191603 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102\": container with ID starting with e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102 not found: ID does not exist" containerID="e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102" Sep 07 09:15:36.191665 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.191632 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102"} err="failed to get container status \"e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102\": rpc error: code = NotFound desc = could not find container \"e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102\": container with ID starting with e90e4df28e63196e69efbdfb74f18961db3fca58fc21b2571a1a3bce3058e102 not found: ID does not exist" Sep 07 09:15:36.202336 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.202269 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w"] Sep 07 09:15:36.206289 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:36.206268 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-da4d4c-f56b9c7bb-wtr9w"] Sep 07 09:15:37.951850 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:37.951814 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="205877fb-a353-4c62-9799-5132a7e8c203" path="/var/lib/kubelet/pods/205877fb-a353-4c62-9799-5132a7e8c203/volumes" Sep 07 09:15:41.853939 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:41.853905 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj"] Sep 07 09:15:41.854337 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:41.854134 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" containerID="cri-o://4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac" gracePeriod=30 Sep 07 09:15:47.139035 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.138950 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:15:47.253248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253219 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-data\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253453 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253261 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-envoy\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253453 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253313 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-credential-socket\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253453 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253404 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-socket\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253436 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-token\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253519 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-podinfo\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253532 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-data" (OuterVolumeSpecName: "istio-data") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:15:47.253647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253585 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-certs\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253612 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d74f0c91-14c6-4740-a2da-f885ef8e287c-istiod-ca-cert\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253905 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253646 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt4zz\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-kube-api-access-bt4zz\") pod \"d74f0c91-14c6-4740-a2da-f885ef8e287c\" (UID: \"d74f0c91-14c6-4740-a2da-f885ef8e287c\") " Sep 07 09:15:47.253905 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.253853 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:15:47.254586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254033 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.254586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254057 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.254586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254235 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:15:47.254586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254408 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d74f0c91-14c6-4740-a2da-f885ef8e287c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:15:47.254586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254524 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:15:47.254930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254700 2558 generic.go:356] "Generic (PLEG): container finished" podID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerID="4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac" exitCode=0 Sep 07 09:15:47.254930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254746 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" event={"ID":"d74f0c91-14c6-4740-a2da-f885ef8e287c","Type":"ContainerDied","Data":"4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac"} Sep 07 09:15:47.254930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254802 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" event={"ID":"d74f0c91-14c6-4740-a2da-f885ef8e287c","Type":"ContainerDied","Data":"1764929b0b2d94a15cf7bf800451ebd09d342ae912a28cf99ec965764ef4cca6"} Sep 07 09:15:47.254930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254833 2558 scope.go:117] "RemoveContainer" containerID="4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac" Sep 07 09:15:47.255140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.254995 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj" Sep 07 09:15:47.256380 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.256353 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:15:47.256544 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.256515 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-kube-api-access-bt4zz" (OuterVolumeSpecName: "kube-api-access-bt4zz") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "kube-api-access-bt4zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:15:47.256647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.256561 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:15:47.256874 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.256786 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-token" (OuterVolumeSpecName: "istio-token") pod "d74f0c91-14c6-4740-a2da-f885ef8e287c" (UID: "d74f0c91-14c6-4740-a2da-f885ef8e287c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:15:47.276890 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.276864 2558 scope.go:117] "RemoveContainer" containerID="d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e" Sep 07 09:15:47.285638 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.285619 2558 scope.go:117] "RemoveContainer" containerID="4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac" Sep 07 09:15:47.285919 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:15:47.285899 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac\": container with ID starting with 4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac not found: ID does not exist" containerID="4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac" Sep 07 09:15:47.286001 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.285932 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac"} err="failed to get container status \"4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac\": rpc error: code = NotFound desc = could not find container \"4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac\": container with ID starting with 4379cf85220b3e36d7c8e1b479c1f2cd2b353394ef319d59c6905dba963c60ac not found: ID does not exist" Sep 07 09:15:47.286001 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.285960 2558 scope.go:117] "RemoveContainer" containerID="d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e" Sep 07 09:15:47.286288 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:15:47.286270 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e\": container with ID starting with d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e not found: ID does not exist" containerID="d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e" Sep 07 09:15:47.286369 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.286294 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e"} err="failed to get container status \"d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e\": rpc error: code = NotFound desc = could not find container \"d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e\": container with ID starting with d96e8d3cf26856cfae60393a877cb20961326fa124ae4799356b8b70c41e2f6e not found: ID does not exist" Sep 07 09:15:47.355241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.355205 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.355241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.355234 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.355241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.355244 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.355509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.355252 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d74f0c91-14c6-4740-a2da-f885ef8e287c-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.355509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.355262 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt4zz\" (UniqueName: \"kubernetes.io/projected/d74f0c91-14c6-4740-a2da-f885ef8e287c-kube-api-access-bt4zz\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.355509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.355272 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.355509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.355281 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d74f0c91-14c6-4740-a2da-f885ef8e287c-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:15:47.578957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.578921 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj"] Sep 07 09:15:47.583166 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.583126 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-da4d4c-openshift-default-56bbf7684b-vtcjj"] Sep 07 09:15:47.951622 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:15:47.951544 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" path="/var/lib/kubelet/pods/d74f0c91-14c6-4740-a2da-f885ef8e287c/volumes" Sep 07 09:16:51.689360 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689320 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt"] Sep 07 09:16:51.689900 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689878 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" Sep 07 09:16:51.689981 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689904 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" Sep 07 09:16:51.689981 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689918 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" containerName="payload-processing" Sep 07 09:16:51.689981 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689926 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" containerName="payload-processing" Sep 07 09:16:51.689981 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689967 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="205877fb-a353-4c62-9799-5132a7e8c203" containerName="payload-pre-processing" Sep 07 09:16:51.689981 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689976 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="205877fb-a353-4c62-9799-5132a7e8c203" containerName="payload-pre-processing" Sep 07 09:16:51.690226 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689989 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41235ffd-849f-47e4-84f3-4f64d6302c68" containerName="revoke-keys" Sep 07 09:16:51.690226 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.689997 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="41235ffd-849f-47e4-84f3-4f64d6302c68" containerName="revoke-keys" Sep 07 09:16:51.690226 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.690092 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" Sep 07 09:16:51.690226 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.690104 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="41235ffd-849f-47e4-84f3-4f64d6302c68" containerName="revoke-keys" Sep 07 09:16:51.690226 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.690116 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf852a5-84ff-4cf9-b3a5-abf05ce94e2e" containerName="payload-processing" Sep 07 09:16:51.690226 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.690128 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="205877fb-a353-4c62-9799-5132a7e8c203" containerName="payload-pre-processing" Sep 07 09:16:51.693343 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.693322 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" Sep 07 09:16:51.695825 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.695802 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:16:51.695925 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.695821 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:16:51.700544 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.700323 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt"] Sep 07 09:16:51.746548 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.746497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-085b77-ck8bt\" (UID: \"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" Sep 07 09:16:51.847576 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.847528 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-085b77-ck8bt\" (UID: \"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" Sep 07 09:16:51.848135 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:51.848117 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-085b77-ck8bt\" (UID: \"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" Sep 07 09:16:52.006078 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:52.005980 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" Sep 07 09:16:52.275764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:52.275734 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt"] Sep 07 09:16:52.276827 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:16:52.276802 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e77fe1_2ae8_4c49_8ad5_fd3b611c764e.slice/crio-4fe2a3c764e11d7edede0784866c8ec3d7f4a4a148f3c432a07c4e9160035b7a WatchSource:0}: Error finding container 4fe2a3c764e11d7edede0784866c8ec3d7f4a4a148f3c432a07c4e9160035b7a: Status 404 returned error can't find the container with id 4fe2a3c764e11d7edede0784866c8ec3d7f4a4a148f3c432a07c4e9160035b7a Sep 07 09:16:52.641537 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:52.641501 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" event={"ID":"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e","Type":"ContainerStarted","Data":"8b60f21087ae64d172745c141c74ab13fbb9c4dd49e86a09ecf9c3d274d2c988"} Sep 07 09:16:52.641712 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:52.641555 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" event={"ID":"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e","Type":"ContainerStarted","Data":"4fe2a3c764e11d7edede0784866c8ec3d7f4a4a148f3c432a07c4e9160035b7a"} Sep 07 09:16:52.655801 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:52.655727 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" podStartSLOduration=1.655710035 podStartE2EDuration="1.655710035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:16:52.65523565 +0000 UTC m=+1071.172026552" watchObservedRunningTime="2026-09-07 09:16:52.655710035 +0000 UTC m=+1071.172500938" Sep 07 09:16:53.648939 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:53.648905 2558 generic.go:356] "Generic (PLEG): container finished" podID="d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e" containerID="8b60f21087ae64d172745c141c74ab13fbb9c4dd49e86a09ecf9c3d274d2c988" exitCode=0 Sep 07 09:16:53.649362 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:53.648986 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" event={"ID":"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e","Type":"ContainerDied","Data":"8b60f21087ae64d172745c141c74ab13fbb9c4dd49e86a09ecf9c3d274d2c988"} Sep 07 09:16:54.787384 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:54.787362 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" Sep 07 09:16:54.872933 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:54.872904 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e-maas-api-service-ca\") pod \"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e\" (UID: \"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e\") " Sep 07 09:16:54.873237 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:54.873215 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e" (UID: "d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:16:54.974511 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:54.974436 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:16:55.661804 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:55.661772 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" Sep 07 09:16:55.661974 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:55.661805 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt" event={"ID":"d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e","Type":"ContainerDied","Data":"4fe2a3c764e11d7edede0784866c8ec3d7f4a4a148f3c432a07c4e9160035b7a"} Sep 07 09:16:55.661974 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:16:55.661852 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fe2a3c764e11d7edede0784866c8ec3d7f4a4a148f3c432a07c4e9160035b7a" Sep 07 09:17:02.442993 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:02.442951 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt"] Sep 07 09:17:02.445230 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:02.445208 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-085b77-ck8bt"] Sep 07 09:17:03.954037 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:03.953998 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e" path="/var/lib/kubelet/pods/d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e/volumes" Sep 07 09:17:07.405627 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.405591 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l"] Sep 07 09:17:07.406019 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.405977 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e" containerName="revoke-keys" Sep 07 09:17:07.406019 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.405987 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e" containerName="revoke-keys" Sep 07 09:17:07.406019 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.405995 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" Sep 07 09:17:07.406019 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.406000 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" Sep 07 09:17:07.406148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.406064 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e77fe1-2ae8-4c49-8ad5-fd3b611c764e" containerName="revoke-keys" Sep 07 09:17:07.406148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.406072 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d74f0c91-14c6-4740-a2da-f885ef8e287c" containerName="istio-proxy" Sep 07 09:17:07.409252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.409236 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.412866 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.412841 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e8e5dd53\"" Sep 07 09:17:07.412994 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.412891 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e8e5dd53-dockercfg-frkgm\"" Sep 07 09:17:07.430120 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.430093 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l"] Sep 07 09:17:07.437871 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.437845 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d"] Sep 07 09:17:07.445447 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.445411 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.455697 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.455672 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d"] Sep 07 09:17:07.492154 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.492126 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-plugins-config-volume\") pod \"payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.492326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.492164 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4njc6\" (UniqueName: \"kubernetes.io/projected/40cfabe8-b057-40a1-b1d3-b28a04e3f334-kube-api-access-4njc6\") pod \"payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.492326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.492194 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40cfabe8-b057-40a1-b1d3-b28a04e3f334-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.492326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.492235 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hqt9\" (UniqueName: \"kubernetes.io/projected/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-kube-api-access-2hqt9\") pod \"payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.593596 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.593563 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-plugins-config-volume\") pod \"payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.593751 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.593608 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4njc6\" (UniqueName: \"kubernetes.io/projected/40cfabe8-b057-40a1-b1d3-b28a04e3f334-kube-api-access-4njc6\") pod \"payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.593751 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.593634 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40cfabe8-b057-40a1-b1d3-b28a04e3f334-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.593886 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.593754 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hqt9\" (UniqueName: \"kubernetes.io/projected/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-kube-api-access-2hqt9\") pod \"payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.594365 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.594347 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40cfabe8-b057-40a1-b1d3-b28a04e3f334-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.594454 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.594362 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-plugins-config-volume\") pod \"payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.601558 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.601533 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hqt9\" (UniqueName: \"kubernetes.io/projected/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-kube-api-access-2hqt9\") pod \"payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.601649 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.601539 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4njc6\" (UniqueName: \"kubernetes.io/projected/40cfabe8-b057-40a1-b1d3-b28a04e3f334-kube-api-access-4njc6\") pod \"payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.719449 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.719338 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:07.759403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.759371 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:07.991121 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:07.991096 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l"] Sep 07 09:17:07.993005 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:17:07.992977 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40cfabe8_b057_40a1_b1d3_b28a04e3f334.slice/crio-4192eb1d042133c12eee69e69f06d6467d910f2ca3ab37b874c5ff4426535cbd WatchSource:0}: Error finding container 4192eb1d042133c12eee69e69f06d6467d910f2ca3ab37b874c5ff4426535cbd: Status 404 returned error can't find the container with id 4192eb1d042133c12eee69e69f06d6467d910f2ca3ab37b874c5ff4426535cbd Sep 07 09:17:08.043614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.043593 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d"] Sep 07 09:17:08.044966 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:17:08.044943 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb1b630_1da4_469f_bc3c_2fb51ef37a39.slice/crio-6080e112e80734888a270714ce2eff46f8f74150bf6ae1a1c34d72e4bb38cb2e WatchSource:0}: Error finding container 6080e112e80734888a270714ce2eff46f8f74150bf6ae1a1c34d72e4bb38cb2e: Status 404 returned error can't find the container with id 6080e112e80734888a270714ce2eff46f8f74150bf6ae1a1c34d72e4bb38cb2e Sep 07 09:17:08.737791 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.737749 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" event={"ID":"40cfabe8-b057-40a1-b1d3-b28a04e3f334","Type":"ContainerStarted","Data":"18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b"} Sep 07 09:17:08.738268 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.737819 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" event={"ID":"40cfabe8-b057-40a1-b1d3-b28a04e3f334","Type":"ContainerStarted","Data":"4192eb1d042133c12eee69e69f06d6467d910f2ca3ab37b874c5ff4426535cbd"} Sep 07 09:17:08.738268 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.737938 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:08.739444 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.739418 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" event={"ID":"0cb1b630-1da4-469f-bc3c-2fb51ef37a39","Type":"ContainerStarted","Data":"8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7"} Sep 07 09:17:08.739570 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.739462 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" event={"ID":"0cb1b630-1da4-469f-bc3c-2fb51ef37a39","Type":"ContainerStarted","Data":"6080e112e80734888a270714ce2eff46f8f74150bf6ae1a1c34d72e4bb38cb2e"} Sep 07 09:17:08.739570 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.739490 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:08.755108 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.755067 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" podStartSLOduration=1.755054276 podStartE2EDuration="1.755054276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:08.752086302 +0000 UTC m=+1087.268877204" watchObservedRunningTime="2026-09-07 09:17:08.755054276 +0000 UTC m=+1087.271845199" Sep 07 09:17:08.767499 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:08.767174 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" podStartSLOduration=1.767157654 podStartE2EDuration="1.767157654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:08.766941732 +0000 UTC m=+1087.283732636" watchObservedRunningTime="2026-09-07 09:17:08.767157654 +0000 UTC m=+1087.283948559" Sep 07 09:17:19.747688 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:19.747657 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:19.748190 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:19.747941 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:37.632139 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.632107 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn"] Sep 07 09:17:37.635691 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.635674 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" Sep 07 09:17:37.638139 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.638091 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:17:37.638478 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.638452 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:17:37.642294 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.642267 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn"] Sep 07 09:17:37.770360 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.770279 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4570814-5075-47da-87f3-555778530a7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn\" (UID: \"b4570814-5075-47da-87f3-555778530a7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" Sep 07 09:17:37.871436 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.871393 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4570814-5075-47da-87f3-555778530a7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn\" (UID: \"b4570814-5075-47da-87f3-555778530a7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" Sep 07 09:17:37.872042 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.872020 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4570814-5075-47da-87f3-555778530a7b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn\" (UID: \"b4570814-5075-47da-87f3-555778530a7b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" Sep 07 09:17:37.948326 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:37.948221 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" Sep 07 09:17:38.217552 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:38.217529 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn"] Sep 07 09:17:38.218852 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:17:38.218822 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4570814_5075_47da_87f3_555778530a7b.slice/crio-bf1674ddf34249e8d694eb7d0908388ac432374b71e68c379e5e86030bb1f7b0 WatchSource:0}: Error finding container bf1674ddf34249e8d694eb7d0908388ac432374b71e68c379e5e86030bb1f7b0: Status 404 returned error can't find the container with id bf1674ddf34249e8d694eb7d0908388ac432374b71e68c379e5e86030bb1f7b0 Sep 07 09:17:38.919134 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:38.919106 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" event={"ID":"b4570814-5075-47da-87f3-555778530a7b","Type":"ContainerStarted","Data":"bfb42d44f898bd4ce102b4b8778fb8bb596e1fbdc30776784116658b2491f2b1"} Sep 07 09:17:38.919491 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:38.919163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" event={"ID":"b4570814-5075-47da-87f3-555778530a7b","Type":"ContainerStarted","Data":"bf1674ddf34249e8d694eb7d0908388ac432374b71e68c379e5e86030bb1f7b0"} Sep 07 09:17:38.934248 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:38.934205 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" podStartSLOduration=1.934190863 podStartE2EDuration="1.934190863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:38.932804441 +0000 UTC m=+1117.449595344" watchObservedRunningTime="2026-09-07 09:17:38.934190863 +0000 UTC m=+1117.450981766" Sep 07 09:17:39.926660 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:39.926624 2558 generic.go:356] "Generic (PLEG): container finished" podID="b4570814-5075-47da-87f3-555778530a7b" containerID="bfb42d44f898bd4ce102b4b8778fb8bb596e1fbdc30776784116658b2491f2b1" exitCode=0 Sep 07 09:17:39.927048 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:39.926708 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" event={"ID":"b4570814-5075-47da-87f3-555778530a7b","Type":"ContainerDied","Data":"bfb42d44f898bd4ce102b4b8778fb8bb596e1fbdc30776784116658b2491f2b1"} Sep 07 09:17:41.068200 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:41.068175 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" Sep 07 09:17:41.101363 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:41.101325 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4570814-5075-47da-87f3-555778530a7b-maas-api-service-ca\") pod \"b4570814-5075-47da-87f3-555778530a7b\" (UID: \"b4570814-5075-47da-87f3-555778530a7b\") " Sep 07 09:17:41.101671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:41.101648 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4570814-5075-47da-87f3-555778530a7b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4570814-5075-47da-87f3-555778530a7b" (UID: "b4570814-5075-47da-87f3-555778530a7b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:17:41.202508 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:41.202424 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4570814-5075-47da-87f3-555778530a7b-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:17:41.949581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:41.949525 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" Sep 07 09:17:41.953005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:41.952970 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn" event={"ID":"b4570814-5075-47da-87f3-555778530a7b","Type":"ContainerDied","Data":"bf1674ddf34249e8d694eb7d0908388ac432374b71e68c379e5e86030bb1f7b0"} Sep 07 09:17:41.953108 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:41.953028 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf1674ddf34249e8d694eb7d0908388ac432374b71e68c379e5e86030bb1f7b0" Sep 07 09:17:48.135293 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.135259 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d"] Sep 07 09:17:48.135774 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.135517 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" podUID="0cb1b630-1da4-469f-bc3c-2fb51ef37a39" containerName="payload-processing" containerID="cri-o://8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7" gracePeriod=30 Sep 07 09:17:48.141659 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.141605 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l"] Sep 07 09:17:48.141871 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.141827 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" podUID="40cfabe8-b057-40a1-b1d3-b28a04e3f334" containerName="payload-pre-processing" containerID="cri-o://18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b" gracePeriod=30 Sep 07 09:17:48.278754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.278411 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn"] Sep 07 09:17:48.281818 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.281792 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e8e5dd53-fn9gn"] Sep 07 09:17:48.422417 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.422390 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:48.429779 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.429761 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:48.465258 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.465226 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hqt9\" (UniqueName: \"kubernetes.io/projected/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-kube-api-access-2hqt9\") pod \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " Sep 07 09:17:48.465441 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.465422 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-plugins-config-volume\") pod \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\" (UID: \"0cb1b630-1da4-469f-bc3c-2fb51ef37a39\") " Sep 07 09:17:48.465503 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.465467 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40cfabe8-b057-40a1-b1d3-b28a04e3f334-plugins-config-volume\") pod \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " Sep 07 09:17:48.465544 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.465503 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4njc6\" (UniqueName: \"kubernetes.io/projected/40cfabe8-b057-40a1-b1d3-b28a04e3f334-kube-api-access-4njc6\") pod \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\" (UID: \"40cfabe8-b057-40a1-b1d3-b28a04e3f334\") " Sep 07 09:17:48.465844 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.465813 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0cb1b630-1da4-469f-bc3c-2fb51ef37a39" (UID: "0cb1b630-1da4-469f-bc3c-2fb51ef37a39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:17:48.465959 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.465869 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40cfabe8-b057-40a1-b1d3-b28a04e3f334-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40cfabe8-b057-40a1-b1d3-b28a04e3f334" (UID: "40cfabe8-b057-40a1-b1d3-b28a04e3f334"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:17:48.467513 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.467489 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-kube-api-access-2hqt9" (OuterVolumeSpecName: "kube-api-access-2hqt9") pod "0cb1b630-1da4-469f-bc3c-2fb51ef37a39" (UID: "0cb1b630-1da4-469f-bc3c-2fb51ef37a39"). InnerVolumeSpecName "kube-api-access-2hqt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:17:48.467625 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.467596 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40cfabe8-b057-40a1-b1d3-b28a04e3f334-kube-api-access-4njc6" (OuterVolumeSpecName: "kube-api-access-4njc6") pod "40cfabe8-b057-40a1-b1d3-b28a04e3f334" (UID: "40cfabe8-b057-40a1-b1d3-b28a04e3f334"). InnerVolumeSpecName "kube-api-access-4njc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:17:48.567033 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.566997 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:17:48.567033 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.567029 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40cfabe8-b057-40a1-b1d3-b28a04e3f334-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:17:48.567033 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.567039 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4njc6\" (UniqueName: \"kubernetes.io/projected/40cfabe8-b057-40a1-b1d3-b28a04e3f334-kube-api-access-4njc6\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:17:48.567262 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.567047 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hqt9\" (UniqueName: \"kubernetes.io/projected/0cb1b630-1da4-469f-bc3c-2fb51ef37a39-kube-api-access-2hqt9\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:17:48.988773 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.988737 2558 generic.go:356] "Generic (PLEG): container finished" podID="40cfabe8-b057-40a1-b1d3-b28a04e3f334" containerID="18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b" exitCode=0 Sep 07 09:17:48.988773 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.988773 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" Sep 07 09:17:48.989086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.988826 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" event={"ID":"40cfabe8-b057-40a1-b1d3-b28a04e3f334","Type":"ContainerDied","Data":"18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b"} Sep 07 09:17:48.989086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.988878 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l" event={"ID":"40cfabe8-b057-40a1-b1d3-b28a04e3f334","Type":"ContainerDied","Data":"4192eb1d042133c12eee69e69f06d6467d910f2ca3ab37b874c5ff4426535cbd"} Sep 07 09:17:48.989086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.988906 2558 scope.go:117] "RemoveContainer" containerID="18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b" Sep 07 09:17:48.990262 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.990242 2558 generic.go:356] "Generic (PLEG): container finished" podID="0cb1b630-1da4-469f-bc3c-2fb51ef37a39" containerID="8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7" exitCode=0 Sep 07 09:17:48.990384 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.990291 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" Sep 07 09:17:48.990384 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.990333 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" event={"ID":"0cb1b630-1da4-469f-bc3c-2fb51ef37a39","Type":"ContainerDied","Data":"8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7"} Sep 07 09:17:48.990461 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.990386 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d" event={"ID":"0cb1b630-1da4-469f-bc3c-2fb51ef37a39","Type":"ContainerDied","Data":"6080e112e80734888a270714ce2eff46f8f74150bf6ae1a1c34d72e4bb38cb2e"} Sep 07 09:17:48.999024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.999009 2558 scope.go:117] "RemoveContainer" containerID="18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b" Sep 07 09:17:48.999351 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:17:48.999292 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b\": container with ID starting with 18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b not found: ID does not exist" containerID="18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b" Sep 07 09:17:48.999461 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.999361 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b"} err="failed to get container status \"18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b\": rpc error: code = NotFound desc = could not find container \"18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b\": container with ID starting with 18589828e56af9169ccf1d1b5713d6606b43c21597bcc18e5396147134725b3b not found: ID does not exist" Sep 07 09:17:48.999461 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:48.999383 2558 scope.go:117] "RemoveContainer" containerID="8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7" Sep 07 09:17:49.007941 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.007924 2558 scope.go:117] "RemoveContainer" containerID="8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7" Sep 07 09:17:49.008252 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:17:49.008221 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7\": container with ID starting with 8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7 not found: ID does not exist" containerID="8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7" Sep 07 09:17:49.008398 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.008262 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7"} err="failed to get container status \"8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7\": rpc error: code = NotFound desc = could not find container \"8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7\": container with ID starting with 8085eb969397cab45a27d89db965f3e3c2f1afd42b67c9b56c66832a437d6be7 not found: ID does not exist" Sep 07 09:17:49.011498 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.011476 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l"] Sep 07 09:17:49.015285 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.015265 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e8e5dd53-7d999d4c58-rvb9l"] Sep 07 09:17:49.024592 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.024572 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d"] Sep 07 09:17:49.028215 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.028195 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e8e5dd53-5b7c4b84cc-25q8d"] Sep 07 09:17:49.952574 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.952541 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb1b630-1da4-469f-bc3c-2fb51ef37a39" path="/var/lib/kubelet/pods/0cb1b630-1da4-469f-bc3c-2fb51ef37a39/volumes" Sep 07 09:17:49.952968 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.952891 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40cfabe8-b057-40a1-b1d3-b28a04e3f334" path="/var/lib/kubelet/pods/40cfabe8-b057-40a1-b1d3-b28a04e3f334/volumes" Sep 07 09:17:49.953184 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:49.953171 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4570814-5075-47da-87f3-555778530a7b" path="/var/lib/kubelet/pods/b4570814-5075-47da-87f3-555778530a7b/volumes" Sep 07 09:17:50.827615 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.827581 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs"] Sep 07 09:17:50.827974 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.827960 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4570814-5075-47da-87f3-555778530a7b" containerName="revoke-keys" Sep 07 09:17:50.828031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.827976 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4570814-5075-47da-87f3-555778530a7b" containerName="revoke-keys" Sep 07 09:17:50.828031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.827997 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40cfabe8-b057-40a1-b1d3-b28a04e3f334" containerName="payload-pre-processing" Sep 07 09:17:50.828031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.828003 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="40cfabe8-b057-40a1-b1d3-b28a04e3f334" containerName="payload-pre-processing" Sep 07 09:17:50.828031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.828014 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb1b630-1da4-469f-bc3c-2fb51ef37a39" containerName="payload-processing" Sep 07 09:17:50.828031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.828020 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb1b630-1da4-469f-bc3c-2fb51ef37a39" containerName="payload-processing" Sep 07 09:17:50.828181 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.828084 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="40cfabe8-b057-40a1-b1d3-b28a04e3f334" containerName="payload-pre-processing" Sep 07 09:17:50.828181 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.828094 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4570814-5075-47da-87f3-555778530a7b" containerName="revoke-keys" Sep 07 09:17:50.828181 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.828101 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb1b630-1da4-469f-bc3c-2fb51ef37a39" containerName="payload-processing" Sep 07 09:17:50.832250 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.832232 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" Sep 07 09:17:50.834938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.834760 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:17:50.835693 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.835671 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:17:50.839315 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.839263 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs"] Sep 07 09:17:50.891058 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.891027 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-614a02-gltrs\" (UID: \"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" Sep 07 09:17:50.992583 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.992544 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-614a02-gltrs\" (UID: \"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" Sep 07 09:17:50.993082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:50.993059 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-614a02-gltrs\" (UID: \"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" Sep 07 09:17:51.145073 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:51.144975 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" Sep 07 09:17:51.419432 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:51.419289 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs"] Sep 07 09:17:51.422126 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:17:51.422097 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f33fd0_a754_4e2b_ba5c_cbdceb0d0b85.slice/crio-c3cc4e943854e050a57c696bc2d6fc79260a86d7ab4fb295fdcb7674633dc3c6 WatchSource:0}: Error finding container c3cc4e943854e050a57c696bc2d6fc79260a86d7ab4fb295fdcb7674633dc3c6: Status 404 returned error can't find the container with id c3cc4e943854e050a57c696bc2d6fc79260a86d7ab4fb295fdcb7674633dc3c6 Sep 07 09:17:52.019430 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:52.019400 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" event={"ID":"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85","Type":"ContainerStarted","Data":"7c5660ea002314eb61be1dc81b62eba0110d5255b86af35e4973e9cdf68a6a10"} Sep 07 09:17:52.019777 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:52.019455 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" event={"ID":"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85","Type":"ContainerStarted","Data":"c3cc4e943854e050a57c696bc2d6fc79260a86d7ab4fb295fdcb7674633dc3c6"} Sep 07 09:17:52.034986 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:52.034943 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" podStartSLOduration=2.034928451 podStartE2EDuration="2.034928451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:52.032497597 +0000 UTC m=+1130.549288492" watchObservedRunningTime="2026-09-07 09:17:52.034928451 +0000 UTC m=+1130.551719354" Sep 07 09:17:53.026950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:53.026916 2558 generic.go:356] "Generic (PLEG): container finished" podID="96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85" containerID="7c5660ea002314eb61be1dc81b62eba0110d5255b86af35e4973e9cdf68a6a10" exitCode=0 Sep 07 09:17:53.027363 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:53.027005 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" event={"ID":"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85","Type":"ContainerDied","Data":"7c5660ea002314eb61be1dc81b62eba0110d5255b86af35e4973e9cdf68a6a10"} Sep 07 09:17:54.162056 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:54.162028 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" Sep 07 09:17:54.223794 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:54.223759 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85-maas-api-service-ca\") pod \"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85\" (UID: \"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85\") " Sep 07 09:17:54.224145 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:54.224121 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85" (UID: "96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:17:54.325106 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:54.325024 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:17:55.040742 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:55.040715 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" Sep 07 09:17:55.040915 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:55.040751 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs" event={"ID":"96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85","Type":"ContainerDied","Data":"c3cc4e943854e050a57c696bc2d6fc79260a86d7ab4fb295fdcb7674633dc3c6"} Sep 07 09:17:55.040915 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:17:55.040796 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3cc4e943854e050a57c696bc2d6fc79260a86d7ab4fb295fdcb7674633dc3c6" Sep 07 09:18:01.351882 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.351839 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4"] Sep 07 09:18:01.352275 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.352085 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" podUID="fab3898d-a417-41c7-aa7b-ee3ad84294e1" containerName="payload-pre-processing" containerID="cri-o://b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb" gracePeriod=30 Sep 07 09:18:01.510781 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.510743 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs"] Sep 07 09:18:01.513112 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.513084 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-614a02-gltrs"] Sep 07 09:18:01.599853 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.599816 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:18:01.698108 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.698024 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbckg\" (UniqueName: \"kubernetes.io/projected/fab3898d-a417-41c7-aa7b-ee3ad84294e1-kube-api-access-bbckg\") pod \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " Sep 07 09:18:01.698108 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.698062 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab3898d-a417-41c7-aa7b-ee3ad84294e1-plugins-config-volume\") pod \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\" (UID: \"fab3898d-a417-41c7-aa7b-ee3ad84294e1\") " Sep 07 09:18:01.698498 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.698474 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fab3898d-a417-41c7-aa7b-ee3ad84294e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fab3898d-a417-41c7-aa7b-ee3ad84294e1" (UID: "fab3898d-a417-41c7-aa7b-ee3ad84294e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:18:01.700077 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.700051 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fab3898d-a417-41c7-aa7b-ee3ad84294e1-kube-api-access-bbckg" (OuterVolumeSpecName: "kube-api-access-bbckg") pod "fab3898d-a417-41c7-aa7b-ee3ad84294e1" (UID: "fab3898d-a417-41c7-aa7b-ee3ad84294e1"). InnerVolumeSpecName "kube-api-access-bbckg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:18:01.799606 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.799572 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbckg\" (UniqueName: \"kubernetes.io/projected/fab3898d-a417-41c7-aa7b-ee3ad84294e1-kube-api-access-bbckg\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:01.799606 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.799601 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fab3898d-a417-41c7-aa7b-ee3ad84294e1-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:01.951762 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:01.951685 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85" path="/var/lib/kubelet/pods/96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85/volumes" Sep 07 09:18:02.082863 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.082830 2558 generic.go:356] "Generic (PLEG): container finished" podID="fab3898d-a417-41c7-aa7b-ee3ad84294e1" containerID="b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb" exitCode=0 Sep 07 09:18:02.082990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.082892 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" event={"ID":"fab3898d-a417-41c7-aa7b-ee3ad84294e1","Type":"ContainerDied","Data":"b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb"} Sep 07 09:18:02.082990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.082931 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" event={"ID":"fab3898d-a417-41c7-aa7b-ee3ad84294e1","Type":"ContainerDied","Data":"2a21cc885929c1b85fa12996ea10843f9d434e5c1c5fc5e5f1c2e124bdbc86e8"} Sep 07 09:18:02.082990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.082951 2558 scope.go:117] "RemoveContainer" containerID="b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb" Sep 07 09:18:02.082990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.082893 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4" Sep 07 09:18:02.091429 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.091413 2558 scope.go:117] "RemoveContainer" containerID="b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb" Sep 07 09:18:02.091689 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:18:02.091671 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb\": container with ID starting with b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb not found: ID does not exist" containerID="b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb" Sep 07 09:18:02.091737 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.091697 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb"} err="failed to get container status \"b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb\": rpc error: code = NotFound desc = could not find container \"b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb\": container with ID starting with b0736f3dd8765e33fac3a5d2e5fdf9d926e5a873261368e1d877dc1cf94b69fb not found: ID does not exist" Sep 07 09:18:02.099080 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.099057 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4"] Sep 07 09:18:02.102647 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:02.102622 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-614a02-56478c7586-jqjz4"] Sep 07 09:18:03.952230 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:03.952196 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fab3898d-a417-41c7-aa7b-ee3ad84294e1" path="/var/lib/kubelet/pods/fab3898d-a417-41c7-aa7b-ee3ad84294e1/volumes" Sep 07 09:18:08.596767 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.596731 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn"] Sep 07 09:18:08.597765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.597735 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85" containerName="revoke-keys" Sep 07 09:18:08.597765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.597765 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85" containerName="revoke-keys" Sep 07 09:18:08.597952 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.597778 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fab3898d-a417-41c7-aa7b-ee3ad84294e1" containerName="payload-pre-processing" Sep 07 09:18:08.597952 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.597786 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fab3898d-a417-41c7-aa7b-ee3ad84294e1" containerName="payload-pre-processing" Sep 07 09:18:08.597952 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.597906 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fab3898d-a417-41c7-aa7b-ee3ad84294e1" containerName="payload-pre-processing" Sep 07 09:18:08.597952 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.597923 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="96f33fd0-a754-4e2b-ba5c-cbdceb0d0b85" containerName="revoke-keys" Sep 07 09:18:08.602712 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.602690 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" Sep 07 09:18:08.605095 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.605073 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:18:08.605412 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.605149 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:18:08.610533 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.610407 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn"] Sep 07 09:18:08.763286 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.763238 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/951ee632-9b0e-4ed1-a024-366c2fae6a04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn\" (UID: \"951ee632-9b0e-4ed1-a024-366c2fae6a04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" Sep 07 09:18:08.864072 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.863974 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/951ee632-9b0e-4ed1-a024-366c2fae6a04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn\" (UID: \"951ee632-9b0e-4ed1-a024-366c2fae6a04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" Sep 07 09:18:08.864662 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.864645 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/951ee632-9b0e-4ed1-a024-366c2fae6a04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn\" (UID: \"951ee632-9b0e-4ed1-a024-366c2fae6a04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" Sep 07 09:18:08.915908 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:08.915870 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" Sep 07 09:18:09.204571 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:09.204548 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn"] Sep 07 09:18:09.206436 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:18:09.206407 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951ee632_9b0e_4ed1_a024_366c2fae6a04.slice/crio-3a5686f92393fe01964038aa4ea94df5ff040c7df39eca823b00b7c791ce704e WatchSource:0}: Error finding container 3a5686f92393fe01964038aa4ea94df5ff040c7df39eca823b00b7c791ce704e: Status 404 returned error can't find the container with id 3a5686f92393fe01964038aa4ea94df5ff040c7df39eca823b00b7c791ce704e Sep 07 09:18:10.133486 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:10.133442 2558 generic.go:356] "Generic (PLEG): container finished" podID="951ee632-9b0e-4ed1-a024-366c2fae6a04" containerID="d9b31f5b858c3a9129adb13104c65f4ccf594780c11516f3e7015613838125c8" exitCode=0 Sep 07 09:18:10.133881 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:10.133503 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" event={"ID":"951ee632-9b0e-4ed1-a024-366c2fae6a04","Type":"ContainerDied","Data":"d9b31f5b858c3a9129adb13104c65f4ccf594780c11516f3e7015613838125c8"} Sep 07 09:18:10.133881 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:10.133563 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" event={"ID":"951ee632-9b0e-4ed1-a024-366c2fae6a04","Type":"ContainerStarted","Data":"3a5686f92393fe01964038aa4ea94df5ff040c7df39eca823b00b7c791ce704e"} Sep 07 09:18:11.286988 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:11.286958 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" Sep 07 09:18:11.388487 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:11.388439 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/951ee632-9b0e-4ed1-a024-366c2fae6a04-maas-api-service-ca\") pod \"951ee632-9b0e-4ed1-a024-366c2fae6a04\" (UID: \"951ee632-9b0e-4ed1-a024-366c2fae6a04\") " Sep 07 09:18:11.388822 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:11.388797 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/951ee632-9b0e-4ed1-a024-366c2fae6a04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "951ee632-9b0e-4ed1-a024-366c2fae6a04" (UID: "951ee632-9b0e-4ed1-a024-366c2fae6a04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:18:11.489849 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:11.489769 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/951ee632-9b0e-4ed1-a024-366c2fae6a04-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:12.145995 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:12.145971 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" Sep 07 09:18:12.146143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:12.146007 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn" event={"ID":"951ee632-9b0e-4ed1-a024-366c2fae6a04","Type":"ContainerDied","Data":"3a5686f92393fe01964038aa4ea94df5ff040c7df39eca823b00b7c791ce704e"} Sep 07 09:18:12.146143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:12.146056 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a5686f92393fe01964038aa4ea94df5ff040c7df39eca823b00b7c791ce704e" Sep 07 09:18:19.369918 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.369879 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk"] Sep 07 09:18:19.370409 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.370113 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" podUID="dab712ab-0543-4057-9c07-dc945c7b454e" containerName="payload-processing" containerID="cri-o://8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124" gracePeriod=30 Sep 07 09:18:19.526479 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.526441 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn"] Sep 07 09:18:19.528841 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.528812 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-3bcf58-5cvzn"] Sep 07 09:18:19.624988 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.624931 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:18:19.673338 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.673286 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ggnk\" (UniqueName: \"kubernetes.io/projected/dab712ab-0543-4057-9c07-dc945c7b454e-kube-api-access-7ggnk\") pod \"dab712ab-0543-4057-9c07-dc945c7b454e\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " Sep 07 09:18:19.673485 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.673346 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dab712ab-0543-4057-9c07-dc945c7b454e-plugins-config-volume\") pod \"dab712ab-0543-4057-9c07-dc945c7b454e\" (UID: \"dab712ab-0543-4057-9c07-dc945c7b454e\") " Sep 07 09:18:19.673740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.673722 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dab712ab-0543-4057-9c07-dc945c7b454e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dab712ab-0543-4057-9c07-dc945c7b454e" (UID: "dab712ab-0543-4057-9c07-dc945c7b454e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:18:19.675289 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.675265 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab712ab-0543-4057-9c07-dc945c7b454e-kube-api-access-7ggnk" (OuterVolumeSpecName: "kube-api-access-7ggnk") pod "dab712ab-0543-4057-9c07-dc945c7b454e" (UID: "dab712ab-0543-4057-9c07-dc945c7b454e"). InnerVolumeSpecName "kube-api-access-7ggnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:18:19.773938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.773904 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ggnk\" (UniqueName: \"kubernetes.io/projected/dab712ab-0543-4057-9c07-dc945c7b454e-kube-api-access-7ggnk\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:19.773938 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.773940 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dab712ab-0543-4057-9c07-dc945c7b454e-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:19.951319 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:19.951211 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="951ee632-9b0e-4ed1-a024-366c2fae6a04" path="/var/lib/kubelet/pods/951ee632-9b0e-4ed1-a024-366c2fae6a04/volumes" Sep 07 09:18:20.194597 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.194562 2558 generic.go:356] "Generic (PLEG): container finished" podID="dab712ab-0543-4057-9c07-dc945c7b454e" containerID="8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124" exitCode=0 Sep 07 09:18:20.194736 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.194605 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" Sep 07 09:18:20.194736 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.194640 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" event={"ID":"dab712ab-0543-4057-9c07-dc945c7b454e","Type":"ContainerDied","Data":"8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124"} Sep 07 09:18:20.194736 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.194691 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk" event={"ID":"dab712ab-0543-4057-9c07-dc945c7b454e","Type":"ContainerDied","Data":"7ea2638701dcb6e35115e84c890226ad901f86d2440aeaa2fccb4b4c00322485"} Sep 07 09:18:20.194736 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.194714 2558 scope.go:117] "RemoveContainer" containerID="8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124" Sep 07 09:18:20.204165 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.204152 2558 scope.go:117] "RemoveContainer" containerID="8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124" Sep 07 09:18:20.204437 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:18:20.204415 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124\": container with ID starting with 8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124 not found: ID does not exist" containerID="8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124" Sep 07 09:18:20.204494 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.204445 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124"} err="failed to get container status \"8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124\": rpc error: code = NotFound desc = could not find container \"8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124\": container with ID starting with 8a730dd9f25a4b55b0f8ab7f8c3bcb527b32b769d107154f99b7b14b862f3124 not found: ID does not exist" Sep 07 09:18:20.214173 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.214152 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk"] Sep 07 09:18:20.218545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:20.218522 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-3bcf58-6b9b8b69f7-rxqtk"] Sep 07 09:18:21.952591 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:21.952560 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab712ab-0543-4057-9c07-dc945c7b454e" path="/var/lib/kubelet/pods/dab712ab-0543-4057-9c07-dc945c7b454e/volumes" Sep 07 09:18:25.942629 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:25.942538 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84"] Sep 07 09:18:25.943073 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:25.942764 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" containerID="cri-o://129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae" gracePeriod=30 Sep 07 09:18:31.098420 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.098386 2558 patch_prober.go:28] interesting pod/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Sep 07 09:18:31.098773 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.098459 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Sep 07 09:18:31.194046 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.194022 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:18:31.263341 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.263232 2558 generic.go:356] "Generic (PLEG): container finished" podID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerID="129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae" exitCode=0 Sep 07 09:18:31.263341 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.263280 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" Sep 07 09:18:31.263577 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.263281 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" event={"ID":"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8","Type":"ContainerDied","Data":"129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae"} Sep 07 09:18:31.263577 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.263425 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84" event={"ID":"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8","Type":"ContainerDied","Data":"1993d8a11ab57c14c1614a6dfe8d8181ca4e52ffc273617f9aa97f5dbbb2ae64"} Sep 07 09:18:31.263577 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.263461 2558 scope.go:117] "RemoveContainer" containerID="129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae" Sep 07 09:18:31.272794 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.272773 2558 scope.go:117] "RemoveContainer" containerID="7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b" Sep 07 09:18:31.275986 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.275969 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-certs\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.275998 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-podinfo\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276022 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-token\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276054 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-envoy\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276225 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276094 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istiod-ca-cert\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276225 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276148 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l92nk\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-kube-api-access-l92nk\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276225 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276185 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-credential-socket\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276407 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276229 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-data\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276407 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276291 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-socket\") pod \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\" (UID: \"4c935805-cca1-42d0-a0d4-f10b6e2c6cc8\") " Sep 07 09:18:31.276561 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276531 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:18:31.276626 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276547 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:18:31.276688 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276663 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.276688 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276682 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.276793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276284 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:18:31.276793 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276758 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:18:31.276899 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.276851 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:18:31.278195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.278170 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:18:31.278318 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.278282 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:18:31.278513 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.278486 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:18:31.278821 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.278790 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-kube-api-access-l92nk" (OuterVolumeSpecName: "kube-api-access-l92nk") pod "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" (UID: "4c935805-cca1-42d0-a0d4-f10b6e2c6cc8"). InnerVolumeSpecName "kube-api-access-l92nk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:18:31.282431 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.282411 2558 scope.go:117] "RemoveContainer" containerID="129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae" Sep 07 09:18:31.282867 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:18:31.282833 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae\": container with ID starting with 129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae not found: ID does not exist" containerID="129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae" Sep 07 09:18:31.283010 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.282867 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae"} err="failed to get container status \"129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae\": rpc error: code = NotFound desc = could not find container \"129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae\": container with ID starting with 129090b467e2a2b14b8d60646cf09d53f670ebd13d1afe81900ada91a2fa14ae not found: ID does not exist" Sep 07 09:18:31.283010 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.282889 2558 scope.go:117] "RemoveContainer" containerID="7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b" Sep 07 09:18:31.283477 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:18:31.283448 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b\": container with ID starting with 7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b not found: ID does not exist" containerID="7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b" Sep 07 09:18:31.283583 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.283486 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b"} err="failed to get container status \"7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b\": rpc error: code = NotFound desc = could not find container \"7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b\": container with ID starting with 7a9b6238b61366e27d8b0935230a4ff62a78a5e75b8e157d1eb9f0d7570ea75b not found: ID does not exist" Sep 07 09:18:31.377516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.377475 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.377516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.377509 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.377516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.377523 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.377764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.377534 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.377764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.377545 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.377764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.377555 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.377764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.377565 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l92nk\" (UniqueName: \"kubernetes.io/projected/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8-kube-api-access-l92nk\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:18:31.586554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.586525 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84"] Sep 07 09:18:31.590656 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.590632 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-3bcf58-openshift-default-65b797cd94-w5w84"] Sep 07 09:18:31.953243 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:31.953169 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" path="/var/lib/kubelet/pods/4c935805-cca1-42d0-a0d4-f10b6e2c6cc8/volumes" Sep 07 09:18:32.006428 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006399 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5"] Sep 07 09:18:32.006801 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006788 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="951ee632-9b0e-4ed1-a024-366c2fae6a04" containerName="revoke-keys" Sep 07 09:18:32.006852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006803 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="951ee632-9b0e-4ed1-a024-366c2fae6a04" containerName="revoke-keys" Sep 07 09:18:32.006852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006818 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab712ab-0543-4057-9c07-dc945c7b454e" containerName="payload-processing" Sep 07 09:18:32.006852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006825 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab712ab-0543-4057-9c07-dc945c7b454e" containerName="payload-processing" Sep 07 09:18:32.006852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006833 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" Sep 07 09:18:32.006852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006840 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" Sep 07 09:18:32.006852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006845 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" Sep 07 09:18:32.007028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006854 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" Sep 07 09:18:32.007028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006918 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="951ee632-9b0e-4ed1-a024-366c2fae6a04" containerName="revoke-keys" Sep 07 09:18:32.007028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006930 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" Sep 07 09:18:32.007028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006940 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c935805-cca1-42d0-a0d4-f10b6e2c6cc8" containerName="istio-proxy" Sep 07 09:18:32.007028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.006950 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab712ab-0543-4057-9c07-dc945c7b454e" containerName="payload-processing" Sep 07 09:18:32.011454 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.011433 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.014024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.014003 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-3dc031\"" Sep 07 09:18:32.021126 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.021056 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5"] Sep 07 09:18:32.049041 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.049010 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq"] Sep 07 09:18:32.052337 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.052320 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.054927 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.054907 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-3dc031-dockercfg-qgh4x\"" Sep 07 09:18:32.063862 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.063840 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq"] Sep 07 09:18:32.083941 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.083918 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a15020df-8c55-4b68-8845-6cfb3dac4b15-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.084023 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.083992 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7435668-73f9-4c80-b9aa-3581dc97cebb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.084074 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.084045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whhgt\" (UniqueName: \"kubernetes.io/projected/a15020df-8c55-4b68-8845-6cfb3dac4b15-kube-api-access-whhgt\") pod \"payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.084118 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.084093 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76w24\" (UniqueName: \"kubernetes.io/projected/e7435668-73f9-4c80-b9aa-3581dc97cebb-kube-api-access-76w24\") pod \"payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.184730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.184696 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7435668-73f9-4c80-b9aa-3581dc97cebb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.185091 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.184756 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whhgt\" (UniqueName: \"kubernetes.io/projected/a15020df-8c55-4b68-8845-6cfb3dac4b15-kube-api-access-whhgt\") pod \"payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.185091 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.184780 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76w24\" (UniqueName: \"kubernetes.io/projected/e7435668-73f9-4c80-b9aa-3581dc97cebb-kube-api-access-76w24\") pod \"payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.185091 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.184803 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a15020df-8c55-4b68-8845-6cfb3dac4b15-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.185466 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.185448 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7435668-73f9-4c80-b9aa-3581dc97cebb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.185511 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.185446 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a15020df-8c55-4b68-8845-6cfb3dac4b15-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.192327 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.192277 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76w24\" (UniqueName: \"kubernetes.io/projected/e7435668-73f9-4c80-b9aa-3581dc97cebb-kube-api-access-76w24\") pod \"payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.192453 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.192401 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whhgt\" (UniqueName: \"kubernetes.io/projected/a15020df-8c55-4b68-8845-6cfb3dac4b15-kube-api-access-whhgt\") pod \"payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.322415 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.322381 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:32.362630 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.362588 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:32.596333 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.596288 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5"] Sep 07 09:18:32.597487 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:18:32.597462 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15020df_8c55_4b68_8845_6cfb3dac4b15.slice/crio-35f7a2d6890c2b1252507d72cbc959b2dc60f89769c5a46c605f122c06709dbb WatchSource:0}: Error finding container 35f7a2d6890c2b1252507d72cbc959b2dc60f89769c5a46c605f122c06709dbb: Status 404 returned error can't find the container with id 35f7a2d6890c2b1252507d72cbc959b2dc60f89769c5a46c605f122c06709dbb Sep 07 09:18:32.649359 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:32.649336 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq"] Sep 07 09:18:32.650543 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:18:32.650520 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7435668_73f9_4c80_b9aa_3581dc97cebb.slice/crio-9995a719070521cadac6f9ee364cdd81669f70e56d6353f9a077951f5126d773 WatchSource:0}: Error finding container 9995a719070521cadac6f9ee364cdd81669f70e56d6353f9a077951f5126d773: Status 404 returned error can't find the container with id 9995a719070521cadac6f9ee364cdd81669f70e56d6353f9a077951f5126d773 Sep 07 09:18:33.277553 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.277517 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" event={"ID":"a15020df-8c55-4b68-8845-6cfb3dac4b15","Type":"ContainerStarted","Data":"0d62fefae190085e928574a733c94970e0a020cb14a1fe0a97c6944bb45ad9dc"} Sep 07 09:18:33.278013 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.277578 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" event={"ID":"a15020df-8c55-4b68-8845-6cfb3dac4b15","Type":"ContainerStarted","Data":"35f7a2d6890c2b1252507d72cbc959b2dc60f89769c5a46c605f122c06709dbb"} Sep 07 09:18:33.278013 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.277709 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:33.279241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.279220 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" event={"ID":"e7435668-73f9-4c80-b9aa-3581dc97cebb","Type":"ContainerStarted","Data":"9f9ce66c4bb750715641d040190f69bf702c0ce18ad6f054054cb4dfc84fbb64"} Sep 07 09:18:33.279371 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.279256 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" event={"ID":"e7435668-73f9-4c80-b9aa-3581dc97cebb","Type":"ContainerStarted","Data":"9995a719070521cadac6f9ee364cdd81669f70e56d6353f9a077951f5126d773"} Sep 07 09:18:33.279371 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.279276 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:33.295514 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.295472 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" podStartSLOduration=2.295458997 podStartE2EDuration="2.295458997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:33.292829005 +0000 UTC m=+1171.809619927" watchObservedRunningTime="2026-09-07 09:18:33.295458997 +0000 UTC m=+1171.812249942" Sep 07 09:18:33.311835 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:33.311776 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" podStartSLOduration=1.3117567239999999 podStartE2EDuration="1.311756724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:33.308593076 +0000 UTC m=+1171.825384007" watchObservedRunningTime="2026-09-07 09:18:33.311756724 +0000 UTC m=+1171.828547628" Sep 07 09:18:37.652852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.652736 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz"] Sep 07 09:18:37.675839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.657157 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.675839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.659565 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-5560ef-openshift-default-dockercfg-x68fj\"" Sep 07 09:18:37.675839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.667898 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz"] Sep 07 09:18:37.737483 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737446 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c99bj\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-kube-api-access-c99bj\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737664 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737521 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istiod-ca-cert\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737664 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737588 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-socket\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737664 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737650 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-envoy\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737781 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737683 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-token\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737781 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-data\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737781 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737757 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-certs\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737781 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737778 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-credential-socket\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.737908 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.737825 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-podinfo\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839189 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839148 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istiod-ca-cert\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839402 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839202 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-socket\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839402 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839241 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-envoy\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839402 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839272 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-token\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839402 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839327 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-data\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839402 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839357 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-certs\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839673 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839418 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-credential-socket\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839673 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839480 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-podinfo\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839673 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839514 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c99bj\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-kube-api-access-c99bj\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839841 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839708 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-socket\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839841 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839757 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-certs\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839928 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istiod-ca-cert\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.839961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.839925 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-credential-socket\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.840119 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.840101 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-data\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.841735 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.841713 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-envoy\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.841864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.841811 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-podinfo\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.846912 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.846889 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-token\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.846994 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.846944 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c99bj\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-kube-api-access-c99bj\") pod \"e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:37.969628 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:37.969547 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:38.246606 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:38.246577 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz"] Sep 07 09:18:38.247894 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:18:38.247860 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3458db0_ee1d_40ec_8bc9_e695a820ce15.slice/crio-39bc57903b102a1d406b9b4d00f3bb59f1c0a38631f52cbddad11c92eb5933a6 WatchSource:0}: Error finding container 39bc57903b102a1d406b9b4d00f3bb59f1c0a38631f52cbddad11c92eb5933a6: Status 404 returned error can't find the container with id 39bc57903b102a1d406b9b4d00f3bb59f1c0a38631f52cbddad11c92eb5933a6 Sep 07 09:18:38.250078 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:38.250034 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:18:38.250165 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:38.250124 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:18:38.250204 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:38.250172 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:18:38.312540 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:38.312513 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" event={"ID":"f3458db0-ee1d-40ec-8bc9-e695a820ce15","Type":"ContainerStarted","Data":"39bc57903b102a1d406b9b4d00f3bb59f1c0a38631f52cbddad11c92eb5933a6"} Sep 07 09:18:39.320680 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:39.320646 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" event={"ID":"f3458db0-ee1d-40ec-8bc9-e695a820ce15","Type":"ContainerStarted","Data":"7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7"} Sep 07 09:18:39.340438 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:39.340388 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podStartSLOduration=2.340370577 podStartE2EDuration="2.340370577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:39.339142198 +0000 UTC m=+1177.855933101" watchObservedRunningTime="2026-09-07 09:18:39.340370577 +0000 UTC m=+1177.857161480" Sep 07 09:18:39.970495 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:39.970463 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:39.975146 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:39.975122 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:40.326026 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:40.325998 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:40.326925 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:40.326907 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:43.044379 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.044349 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw"] Sep 07 09:18:43.048071 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.048052 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.050572 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.050553 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-5560ef-dockercfg-zlmzk\"" Sep 07 09:18:43.050690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.050590 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-5560ef\"" Sep 07 09:18:43.059481 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.059455 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw"] Sep 07 09:18:43.191400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.191362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71b023f-f081-4224-a8fc-567f535c8ad8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.191578 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.191489 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm2cq\" (UniqueName: \"kubernetes.io/projected/d71b023f-f081-4224-a8fc-567f535c8ad8-kube-api-access-hm2cq\") pod \"payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.292962 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.292918 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm2cq\" (UniqueName: \"kubernetes.io/projected/d71b023f-f081-4224-a8fc-567f535c8ad8-kube-api-access-hm2cq\") pod \"payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.293143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.293016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71b023f-f081-4224-a8fc-567f535c8ad8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.293688 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.293669 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71b023f-f081-4224-a8fc-567f535c8ad8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.300614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.300559 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm2cq\" (UniqueName: \"kubernetes.io/projected/d71b023f-f081-4224-a8fc-567f535c8ad8-kube-api-access-hm2cq\") pod \"payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.360436 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.360410 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:43.633001 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:43.632970 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw"] Sep 07 09:18:43.634287 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:18:43.634262 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71b023f_f081_4224_a8fc_567f535c8ad8.slice/crio-5b2c5798d4693cdb082f3662adeb1f4aad809cb9b6b39e93889452a2d29c22b3 WatchSource:0}: Error finding container 5b2c5798d4693cdb082f3662adeb1f4aad809cb9b6b39e93889452a2d29c22b3: Status 404 returned error can't find the container with id 5b2c5798d4693cdb082f3662adeb1f4aad809cb9b6b39e93889452a2d29c22b3 Sep 07 09:18:44.287773 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:44.287743 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:18:44.288138 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:44.287812 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:18:44.353841 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:44.353799 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" event={"ID":"d71b023f-f081-4224-a8fc-567f535c8ad8","Type":"ContainerStarted","Data":"b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e"} Sep 07 09:18:44.354027 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:44.353859 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" event={"ID":"d71b023f-f081-4224-a8fc-567f535c8ad8","Type":"ContainerStarted","Data":"5b2c5798d4693cdb082f3662adeb1f4aad809cb9b6b39e93889452a2d29c22b3"} Sep 07 09:18:44.354027 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:44.353945 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:44.369452 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:44.369287 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" podStartSLOduration=1.369267423 podStartE2EDuration="1.369267423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:44.368559063 +0000 UTC m=+1182.885349969" watchObservedRunningTime="2026-09-07 09:18:44.369267423 +0000 UTC m=+1182.886058329" Sep 07 09:18:55.362165 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:55.362126 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:18:55.422689 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:55.422653 2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:43590->10.133.0.65:15021: read: connection reset by peer" start-of-body= Sep 07 09:18:55.422833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:55.422725 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": read tcp 10.133.0.2:43590->10.133.0.65:15021: read: connection reset by peer" Sep 07 09:18:56.436702 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:56.436667 2558 generic.go:356] "Generic (PLEG): container finished" podID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerID="7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7" exitCode=0 Sep 07 09:18:56.437183 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:56.436744 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" event={"ID":"f3458db0-ee1d-40ec-8bc9-e695a820ce15","Type":"ContainerDied","Data":"7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7"} Sep 07 09:18:56.437274 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:56.437257 2558 scope.go:117] "RemoveContainer" containerID="7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7" Sep 07 09:18:56.438350 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:56.438285 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:18:56.438441 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:56.438389 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:18:56.438441 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:56.438425 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:18:57.447479 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:57.447447 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" event={"ID":"f3458db0-ee1d-40ec-8bc9-e695a820ce15","Type":"ContainerStarted","Data":"ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277"} Sep 07 09:18:57.970356 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:57.970329 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:18:57.970623 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:57.970604 2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 07 09:18:57.970675 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:57.970658 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 07 09:18:58.970733 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:58.970695 2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 07 09:18:58.971114 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:58.970764 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 07 09:18:59.970389 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:59.970356 2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 07 09:18:59.970570 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:18:59.970431 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 07 09:19:00.970622 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:00.970587 2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 07 09:19:00.970990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:00.970640 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 07 09:19:01.250215 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.250138 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78"] Sep 07 09:19:01.253555 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.253538 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.256166 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.256136 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-34de95-dockercfg-mnl2w\"" Sep 07 09:19:01.256462 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.256446 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-34de95\"" Sep 07 09:19:01.263064 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.263039 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78"] Sep 07 09:19:01.269637 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.269612 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmv2x\" (UniqueName: \"kubernetes.io/projected/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-kube-api-access-cmv2x\") pod \"payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.269746 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.269668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.370951 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.370910 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.371141 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.371075 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmv2x\" (UniqueName: \"kubernetes.io/projected/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-kube-api-access-cmv2x\") pod \"payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.371917 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.371889 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.380231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.380201 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmv2x\" (UniqueName: \"kubernetes.io/projected/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-kube-api-access-cmv2x\") pod \"payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.565859 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.565820 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:01.858516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.858440 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78"] Sep 07 09:19:01.970031 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.969995 2558 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Sep 07 09:19:01.970211 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.970041 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Sep 07 09:19:01.990107 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.990079 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:19:01.991684 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:01.991666 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:19:02.481887 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:02.481815 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" event={"ID":"a7da42f5-56c3-4cc6-826c-c558bec8ff5f","Type":"ContainerStarted","Data":"846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122"} Sep 07 09:19:02.482095 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:02.481898 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" event={"ID":"a7da42f5-56c3-4cc6-826c-c558bec8ff5f","Type":"ContainerStarted","Data":"a2aa4f184a96c56c815b0bfe32accb78808b16f74a072eeb3227a96ebd673e4d"} Sep 07 09:19:02.482095 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:02.481972 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:02.498803 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:02.498735 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" podStartSLOduration=1.4987142549999999 podStartE2EDuration="1.498714255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:02.495997335 +0000 UTC m=+1201.012788239" watchObservedRunningTime="2026-09-07 09:19:02.498714255 +0000 UTC m=+1201.015505159" Sep 07 09:19:02.974030 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:02.974001 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:19:02.974314 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:02.974277 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:19:02.974832 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:02.974815 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:19:13.489873 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:13.489840 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:19:19.799054 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.799018 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w"] Sep 07 09:19:19.803952 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.803931 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.806275 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.806252 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-12fc0fc8-openshift-default-dockercfg-xvsrt\"" Sep 07 09:19:19.815571 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.815548 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w"] Sep 07 09:19:19.941648 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941613 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-token\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.941830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941663 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwd9m\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-kube-api-access-dwd9m\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.941830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941697 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-socket\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.941830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941719 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istiod-ca-cert\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.941830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941742 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-envoy\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.942004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941838 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-credential-socket\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.942004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941872 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-podinfo\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.942004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941890 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-data\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:19.942004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:19.941955 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-certs\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.042577 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042549 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-certs\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.042753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042612 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-token\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.042753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042638 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwd9m\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-kube-api-access-dwd9m\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.042753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042658 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-socket\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.042753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042672 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istiod-ca-cert\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.042753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042692 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-envoy\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.042753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042738 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-credential-socket\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.043079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042765 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-podinfo\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.043079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042783 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-data\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.043079 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.042985 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-certs\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.043396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.043372 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-socket\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.043789 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.043766 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-data\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.043940 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.043621 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-credential-socket\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.044435 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.044407 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istiod-ca-cert\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.050354 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.046764 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-envoy\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.050354 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.047777 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-podinfo\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.051169 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.051145 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwd9m\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-kube-api-access-dwd9m\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.052599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.052577 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-token\") pod \"e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.114516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.114480 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:20.392990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.392964 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w"] Sep 07 09:19:20.393561 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:19:20.393534 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4bf15ef_139b_47ed_8944_b24d28f0ea3d.slice/crio-5a445f0933150c75b758ea8b078cd88c0241d37717030b774efa71d061bd4400 WatchSource:0}: Error finding container 5a445f0933150c75b758ea8b078cd88c0241d37717030b774efa71d061bd4400: Status 404 returned error can't find the container with id 5a445f0933150c75b758ea8b078cd88c0241d37717030b774efa71d061bd4400 Sep 07 09:19:20.395894 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.395839 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:19:20.395961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.395939 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:19:20.395998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.395980 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:19:20.593162 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.593123 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" event={"ID":"d4bf15ef-139b-47ed-8944-b24d28f0ea3d","Type":"ContainerStarted","Data":"c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177"} Sep 07 09:19:20.593383 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.593182 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" event={"ID":"d4bf15ef-139b-47ed-8944-b24d28f0ea3d","Type":"ContainerStarted","Data":"5a445f0933150c75b758ea8b078cd88c0241d37717030b774efa71d061bd4400"} Sep 07 09:19:20.613917 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:20.613866 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podStartSLOduration=1.613850444 podStartE2EDuration="1.613850444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:20.61109991 +0000 UTC m=+1219.127890827" watchObservedRunningTime="2026-09-07 09:19:20.613850444 +0000 UTC m=+1219.130641357" Sep 07 09:19:21.115624 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:21.115586 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:21.120681 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:21.120657 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:21.599320 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:21.599266 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:21.600120 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:21.600103 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:24.340941 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.340911 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st"] Sep 07 09:19:24.344562 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.344544 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.346947 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.346918 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-12fc0fc8-dockercfg-htqg6\"" Sep 07 09:19:24.347077 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.346987 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-12fc0fc8\"" Sep 07 09:19:24.355232 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.355208 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st"] Sep 07 09:19:24.485388 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.485353 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc5cc722-15b5-4f62-989a-6295798edbae-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.485580 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.485406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjtsl\" (UniqueName: \"kubernetes.io/projected/bc5cc722-15b5-4f62-989a-6295798edbae-kube-api-access-kjtsl\") pod \"payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.586466 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.586431 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc5cc722-15b5-4f62-989a-6295798edbae-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.586640 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.586490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjtsl\" (UniqueName: \"kubernetes.io/projected/bc5cc722-15b5-4f62-989a-6295798edbae-kube-api-access-kjtsl\") pod \"payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.587140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.587114 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc5cc722-15b5-4f62-989a-6295798edbae-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.594625 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.594571 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjtsl\" (UniqueName: \"kubernetes.io/projected/bc5cc722-15b5-4f62-989a-6295798edbae-kube-api-access-kjtsl\") pod \"payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.656365 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.656338 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:24.938747 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:24.938708 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st"] Sep 07 09:19:24.939993 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:19:24.939971 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5cc722_15b5_4f62_989a_6295798edbae.slice/crio-ee175cdef53eba89c1cf64239b2a961bc6ed30c1f1495f5ad3554bb555fbd44c WatchSource:0}: Error finding container ee175cdef53eba89c1cf64239b2a961bc6ed30c1f1495f5ad3554bb555fbd44c: Status 404 returned error can't find the container with id ee175cdef53eba89c1cf64239b2a961bc6ed30c1f1495f5ad3554bb555fbd44c Sep 07 09:19:25.625487 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:25.625440 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" event={"ID":"bc5cc722-15b5-4f62-989a-6295798edbae","Type":"ContainerStarted","Data":"95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7"} Sep 07 09:19:25.625881 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:25.625527 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" event={"ID":"bc5cc722-15b5-4f62-989a-6295798edbae","Type":"ContainerStarted","Data":"ee175cdef53eba89c1cf64239b2a961bc6ed30c1f1495f5ad3554bb555fbd44c"} Sep 07 09:19:25.625881 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:25.625549 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:25.642442 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:25.642394 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" podStartSLOduration=1.642379077 podStartE2EDuration="1.642379077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:25.640654364 +0000 UTC m=+1224.157445266" watchObservedRunningTime="2026-09-07 09:19:25.642379077 +0000 UTC m=+1224.159170022" Sep 07 09:19:36.634219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:36.634183 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:19:37.641444 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:37.641406 2558 patch_prober.go:28] interesting pod/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 07 09:19:37.641825 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:37.641462 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 09:19:38.706496 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:38.706412 2558 generic.go:356] "Generic (PLEG): container finished" podID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerID="c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177" exitCode=0 Sep 07 09:19:38.706496 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:38.706485 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" event={"ID":"d4bf15ef-139b-47ed-8944-b24d28f0ea3d","Type":"ContainerDied","Data":"c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177"} Sep 07 09:19:38.706966 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:38.706951 2558 scope.go:117] "RemoveContainer" containerID="c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177" Sep 07 09:19:38.708140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:38.708114 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:19:38.708196 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:38.708180 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:19:38.708231 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:38.708210 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:19:39.714940 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:39.714903 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" event={"ID":"d4bf15ef-139b-47ed-8944-b24d28f0ea3d","Type":"ContainerStarted","Data":"3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418"} Sep 07 09:19:40.114873 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:40.114840 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:40.115051 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:40.115029 2558 patch_prober.go:28] interesting pod/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 07 09:19:40.115096 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:40.115073 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 07 09:19:41.115604 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:41.115562 2558 patch_prober.go:28] interesting pod/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 07 09:19:41.116096 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:41.115627 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 07 09:19:42.114925 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:42.114893 2558 patch_prober.go:28] interesting pod/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 07 09:19:42.115101 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:42.114953 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 07 09:19:43.115615 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:43.115581 2558 patch_prober.go:28] interesting pod/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 07 09:19:43.116195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:43.116166 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 07 09:19:44.115154 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:44.115121 2558 patch_prober.go:28] interesting pod/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Sep 07 09:19:44.115396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:44.115175 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Sep 07 09:19:45.285129 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:45.285083 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:45.285628 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:45.285499 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:45.286404 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:45.286383 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:19:57.801885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.801783 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc"] Sep 07 09:19:57.806368 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.806347 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:57.808868 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.808847 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-12fc0fc8\"/\"kube-root-ca.crt\"" Sep 07 09:19:57.809055 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.808899 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-12fc0fc8\"/\"openshift-service-ca.crt\"" Sep 07 09:19:57.810277 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.810257 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-12fc0fc8\"/\"default-dockercfg-dwh4f\"" Sep 07 09:19:57.810578 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.810558 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-12fc0fc8\"/\"e2e-lifecycle-model-12fc0fc8-kserve-self-signed-certs\"" Sep 07 09:19:57.813551 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.813526 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc"] Sep 07 09:19:57.910205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.910167 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-dshm\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:57.910406 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.910217 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-model-cache\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:57.910406 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.910371 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b32765e-0d73-40ff-be76-07d589cc5f77-tls-certs\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:57.910514 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.910406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-home\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:57.910514 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.910435 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6vkm\" (UniqueName: \"kubernetes.io/projected/1b32765e-0d73-40ff-be76-07d589cc5f77-kube-api-access-z6vkm\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:57.910514 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:57.910490 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-tmp-dir\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.012044 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.011996 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b32765e-0d73-40ff-be76-07d589cc5f77-tls-certs\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.012810 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.012786 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-home\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.013002 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.012985 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6vkm\" (UniqueName: \"kubernetes.io/projected/1b32765e-0d73-40ff-be76-07d589cc5f77-kube-api-access-z6vkm\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.013131 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.013117 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-tmp-dir\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.013239 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.013227 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-dshm\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.013380 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.013365 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-model-cache\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.013864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.013843 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-model-cache\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.014221 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.014203 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-home\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.014871 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.014849 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-tmp-dir\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.018220 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.018190 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-dshm\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.018857 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.018814 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b32765e-0d73-40ff-be76-07d589cc5f77-tls-certs\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.024545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.024522 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6vkm\" (UniqueName: \"kubernetes.io/projected/1b32765e-0d73-40ff-be76-07d589cc5f77-kube-api-access-z6vkm\") pod \"e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.116516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.116416 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:19:58.393219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.393188 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc"] Sep 07 09:19:58.394137 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:19:58.394106 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b32765e_0d73_40ff_be76_07d589cc5f77.slice/crio-2594adcd1c394fc83c027a126a49a080d0a4f0233fdc4ace544a08bbc86da016 WatchSource:0}: Error finding container 2594adcd1c394fc83c027a126a49a080d0a4f0233fdc4ace544a08bbc86da016: Status 404 returned error can't find the container with id 2594adcd1c394fc83c027a126a49a080d0a4f0233fdc4ace544a08bbc86da016 Sep 07 09:19:58.842485 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.842451 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" event={"ID":"1b32765e-0d73-40ff-be76-07d589cc5f77","Type":"ContainerStarted","Data":"3da5eece510c906335154423e459b84e53f5fd5bdf2897548001c7dd9f9935eb"} Sep 07 09:19:58.842485 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.842488 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" event={"ID":"1b32765e-0d73-40ff-be76-07d589cc5f77","Type":"ContainerStarted","Data":"2594adcd1c394fc83c027a126a49a080d0a4f0233fdc4ace544a08bbc86da016"} Sep 07 09:19:58.860744 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:19:58.860692 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" podStartSLOduration=1.860672895 podStartE2EDuration="1.860672895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:58.858174132 +0000 UTC m=+1257.374965035" watchObservedRunningTime="2026-09-07 09:19:58.860672895 +0000 UTC m=+1257.377463797" Sep 07 09:20:02.066195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:02.066165 2558 scope.go:117] "RemoveContainer" containerID="1e25e1bf72820c75edfcacaae8e6f0918791ece1d928a65a48517debf502a345" Sep 07 09:20:08.117141 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:08.117104 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:20:08.117664 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:08.117200 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:20:08.130092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:08.130069 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:20:08.911427 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:08.911398 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:20:47.550178 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.550145 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq"] Sep 07 09:20:47.553886 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.553869 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" Sep 07 09:20:47.556354 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.556336 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:20:47.556445 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.556415 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:20:47.561744 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.561373 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq"] Sep 07 09:20:47.701733 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.701698 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq\" (UID: \"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" Sep 07 09:20:47.803226 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.803143 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq\" (UID: \"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" Sep 07 09:20:47.803785 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.803767 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq\" (UID: \"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" Sep 07 09:20:47.865950 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:47.865924 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" Sep 07 09:20:48.136939 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:48.136901 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq"] Sep 07 09:20:48.138093 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:20:48.138069 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6cc3743_1f2e_48e2_bd4a_ea8dcb3a9874.slice/crio-61c92ffc9636cbdf0182a101973eed357749b658751ebe4a9690f57407760ba3 WatchSource:0}: Error finding container 61c92ffc9636cbdf0182a101973eed357749b658751ebe4a9690f57407760ba3: Status 404 returned error can't find the container with id 61c92ffc9636cbdf0182a101973eed357749b658751ebe4a9690f57407760ba3 Sep 07 09:20:49.143578 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:49.143492 2558 generic.go:356] "Generic (PLEG): container finished" podID="f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874" containerID="edb7d54a6f83c48be4b050d0fd7c2911180d939547cf939d37e7db01f7e91036" exitCode=0 Sep 07 09:20:49.143925 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:49.143575 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" event={"ID":"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874","Type":"ContainerDied","Data":"edb7d54a6f83c48be4b050d0fd7c2911180d939547cf939d37e7db01f7e91036"} Sep 07 09:20:49.143925 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:49.143630 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" event={"ID":"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874","Type":"ContainerStarted","Data":"61c92ffc9636cbdf0182a101973eed357749b658751ebe4a9690f57407760ba3"} Sep 07 09:20:50.290567 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:50.290544 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" Sep 07 09:20:50.426774 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:50.426668 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874-maas-api-service-ca\") pod \"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874\" (UID: \"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874\") " Sep 07 09:20:50.427025 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:50.427002 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874" (UID: "f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:20:50.527659 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:50.527623 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:20:51.157321 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:51.157272 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" Sep 07 09:20:51.157502 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:51.157343 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq" event={"ID":"f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874","Type":"ContainerDied","Data":"61c92ffc9636cbdf0182a101973eed357749b658751ebe4a9690f57407760ba3"} Sep 07 09:20:51.157502 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:20:51.157418 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61c92ffc9636cbdf0182a101973eed357749b658751ebe4a9690f57407760ba3" Sep 07 09:21:02.087723 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:02.087685 2558 scope.go:117] "RemoveContainer" containerID="32a53f3a685747e28fc91d7bae5a4d1c5777d83c7d6976e9d7abd5adf5d48707" Sep 07 09:21:02.097023 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:02.097001 2558 scope.go:117] "RemoveContainer" containerID="4f46d30303631336c88151f27f8c4c9f7d31cfc885e41f4ea29457825b368b3e" Sep 07 09:21:03.127810 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.127776 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st"] Sep 07 09:21:03.128289 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.128018 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" podUID="bc5cc722-15b5-4f62-989a-6295798edbae" containerName="payload-pre-processing" containerID="cri-o://95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7" gracePeriod=30 Sep 07 09:21:03.291021 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.290986 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq"] Sep 07 09:21:03.293574 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.293548 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-12fc0fc8-fpznq"] Sep 07 09:21:03.427436 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.427414 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:21:03.550454 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.550417 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc5cc722-15b5-4f62-989a-6295798edbae-plugins-config-volume\") pod \"bc5cc722-15b5-4f62-989a-6295798edbae\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " Sep 07 09:21:03.550638 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.550479 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjtsl\" (UniqueName: \"kubernetes.io/projected/bc5cc722-15b5-4f62-989a-6295798edbae-kube-api-access-kjtsl\") pod \"bc5cc722-15b5-4f62-989a-6295798edbae\" (UID: \"bc5cc722-15b5-4f62-989a-6295798edbae\") " Sep 07 09:21:03.550868 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.550840 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc5cc722-15b5-4f62-989a-6295798edbae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc5cc722-15b5-4f62-989a-6295798edbae" (UID: "bc5cc722-15b5-4f62-989a-6295798edbae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:21:03.552717 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.552696 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5cc722-15b5-4f62-989a-6295798edbae-kube-api-access-kjtsl" (OuterVolumeSpecName: "kube-api-access-kjtsl") pod "bc5cc722-15b5-4f62-989a-6295798edbae" (UID: "bc5cc722-15b5-4f62-989a-6295798edbae"). InnerVolumeSpecName "kube-api-access-kjtsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:21:03.651870 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.651780 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc5cc722-15b5-4f62-989a-6295798edbae-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:03.651870 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.651807 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjtsl\" (UniqueName: \"kubernetes.io/projected/bc5cc722-15b5-4f62-989a-6295798edbae-kube-api-access-kjtsl\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:03.952280 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:03.952192 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874" path="/var/lib/kubelet/pods/f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874/volumes" Sep 07 09:21:04.241339 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.241233 2558 generic.go:356] "Generic (PLEG): container finished" podID="bc5cc722-15b5-4f62-989a-6295798edbae" containerID="95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7" exitCode=0 Sep 07 09:21:04.241339 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.241268 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" event={"ID":"bc5cc722-15b5-4f62-989a-6295798edbae","Type":"ContainerDied","Data":"95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7"} Sep 07 09:21:04.241339 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.241280 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" Sep 07 09:21:04.241852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.241341 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st" event={"ID":"bc5cc722-15b5-4f62-989a-6295798edbae","Type":"ContainerDied","Data":"ee175cdef53eba89c1cf64239b2a961bc6ed30c1f1495f5ad3554bb555fbd44c"} Sep 07 09:21:04.241852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.241364 2558 scope.go:117] "RemoveContainer" containerID="95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7" Sep 07 09:21:04.251005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.250976 2558 scope.go:117] "RemoveContainer" containerID="95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7" Sep 07 09:21:04.251279 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:21:04.251261 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7\": container with ID starting with 95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7 not found: ID does not exist" containerID="95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7" Sep 07 09:21:04.251362 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.251321 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7"} err="failed to get container status \"95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7\": rpc error: code = NotFound desc = could not find container \"95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7\": container with ID starting with 95e95792afdd6e9036c0a931514dd747d62c20e56babebe3fed6a700931403c7 not found: ID does not exist" Sep 07 09:21:04.258669 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.258647 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st"] Sep 07 09:21:04.262096 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:04.262074 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-12fc0fc8-5869689bcd-6s4st"] Sep 07 09:21:05.952143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:05.952108 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc5cc722-15b5-4f62-989a-6295798edbae" path="/var/lib/kubelet/pods/bc5cc722-15b5-4f62-989a-6295798edbae/volumes" Sep 07 09:21:11.164040 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:11.164004 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc"] Sep 07 09:21:26.184652 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.184561 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" podUID="1b32765e-0d73-40ff-be76-07d589cc5f77" containerName="main" containerID="cri-o://3da5eece510c906335154423e459b84e53f5fd5bdf2897548001c7dd9f9935eb" gracePeriod=45 Sep 07 09:21:26.380980 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.380948 2558 generic.go:356] "Generic (PLEG): container finished" podID="1b32765e-0d73-40ff-be76-07d589cc5f77" containerID="3da5eece510c906335154423e459b84e53f5fd5bdf2897548001c7dd9f9935eb" exitCode=0 Sep 07 09:21:26.381154 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.381026 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" event={"ID":"1b32765e-0d73-40ff-be76-07d589cc5f77","Type":"ContainerDied","Data":"3da5eece510c906335154423e459b84e53f5fd5bdf2897548001c7dd9f9935eb"} Sep 07 09:21:26.439990 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.439929 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:21:26.571552 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571522 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-home\") pod \"1b32765e-0d73-40ff-be76-07d589cc5f77\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " Sep 07 09:21:26.571740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571573 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-dshm\") pod \"1b32765e-0d73-40ff-be76-07d589cc5f77\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " Sep 07 09:21:26.571740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571604 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6vkm\" (UniqueName: \"kubernetes.io/projected/1b32765e-0d73-40ff-be76-07d589cc5f77-kube-api-access-z6vkm\") pod \"1b32765e-0d73-40ff-be76-07d589cc5f77\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " Sep 07 09:21:26.571740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571655 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-tmp-dir\") pod \"1b32765e-0d73-40ff-be76-07d589cc5f77\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " Sep 07 09:21:26.571740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571683 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b32765e-0d73-40ff-be76-07d589cc5f77-tls-certs\") pod \"1b32765e-0d73-40ff-be76-07d589cc5f77\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " Sep 07 09:21:26.571740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571712 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-model-cache\") pod \"1b32765e-0d73-40ff-be76-07d589cc5f77\" (UID: \"1b32765e-0d73-40ff-be76-07d589cc5f77\") " Sep 07 09:21:26.572022 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571814 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-home" (OuterVolumeSpecName: "home") pod "1b32765e-0d73-40ff-be76-07d589cc5f77" (UID: "1b32765e-0d73-40ff-be76-07d589cc5f77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:26.572022 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.571883 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b32765e-0d73-40ff-be76-07d589cc5f77" (UID: "1b32765e-0d73-40ff-be76-07d589cc5f77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:26.572022 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.572007 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b32765e-0d73-40ff-be76-07d589cc5f77" (UID: "1b32765e-0d73-40ff-be76-07d589cc5f77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:26.572130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.572115 2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-tmp-dir\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:26.572171 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.572135 2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-model-cache\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:26.572171 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.572149 2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-home\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:26.573824 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.573803 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-dshm" (OuterVolumeSpecName: "dshm") pod "1b32765e-0d73-40ff-be76-07d589cc5f77" (UID: "1b32765e-0d73-40ff-be76-07d589cc5f77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:26.573910 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.573864 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b32765e-0d73-40ff-be76-07d589cc5f77-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b32765e-0d73-40ff-be76-07d589cc5f77" (UID: "1b32765e-0d73-40ff-be76-07d589cc5f77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 07 09:21:26.573979 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.573959 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b32765e-0d73-40ff-be76-07d589cc5f77-kube-api-access-z6vkm" (OuterVolumeSpecName: "kube-api-access-z6vkm") pod "1b32765e-0d73-40ff-be76-07d589cc5f77" (UID: "1b32765e-0d73-40ff-be76-07d589cc5f77"). InnerVolumeSpecName "kube-api-access-z6vkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:21:26.678022 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.677983 2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b32765e-0d73-40ff-be76-07d589cc5f77-dshm\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:26.678227 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.678039 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6vkm\" (UniqueName: \"kubernetes.io/projected/1b32765e-0d73-40ff-be76-07d589cc5f77-kube-api-access-z6vkm\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:26.678227 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:26.678051 2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b32765e-0d73-40ff-be76-07d589cc5f77-tls-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:27.389219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:27.389178 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" event={"ID":"1b32765e-0d73-40ff-be76-07d589cc5f77","Type":"ContainerDied","Data":"2594adcd1c394fc83c027a126a49a080d0a4f0233fdc4ace544a08bbc86da016"} Sep 07 09:21:27.389219 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:27.389203 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc" Sep 07 09:21:27.389717 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:27.389236 2558 scope.go:117] "RemoveContainer" containerID="3da5eece510c906335154423e459b84e53f5fd5bdf2897548001c7dd9f9935eb" Sep 07 09:21:27.411584 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:27.411519 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc"] Sep 07 09:21:27.415229 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:27.415203 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-12fc0fc8/e2e-lifecycle-model-12fc0fc8-kserve-6dbbc95b5f-cf8zc"] Sep 07 09:21:27.953047 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:27.953008 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b32765e-0d73-40ff-be76-07d589cc5f77" path="/var/lib/kubelet/pods/1b32765e-0d73-40ff-be76-07d589cc5f77/volumes" Sep 07 09:21:53.822519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:53.822485 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w"] Sep 07 09:21:53.823010 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:53.822726 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" containerID="cri-o://3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418" gracePeriod=30 Sep 07 09:21:59.072901 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.072871 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:21:59.192956 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.192879 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-token\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.192956 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.192921 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-certs\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.192956 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.192954 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istiod-ca-cert\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.193241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.192997 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-data\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.193241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193027 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwd9m\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-kube-api-access-dwd9m\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.193241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193052 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-socket\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.193241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193105 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-podinfo\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.193241 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193148 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-envoy\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.193546 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193255 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-credential-socket\") pod \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\" (UID: \"d4bf15ef-139b-47ed-8944-b24d28f0ea3d\") " Sep 07 09:21:59.193546 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193321 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:59.193546 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193341 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:59.193546 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193432 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:21:59.193546 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193458 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:59.193767 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193650 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:59.193767 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193724 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.193767 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193743 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.193767 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193752 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.193767 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193760 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.193767 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.193767 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.195087 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.195059 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:21:59.195257 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.195239 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:21:59.195542 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.195518 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-kube-api-access-dwd9m" (OuterVolumeSpecName: "kube-api-access-dwd9m") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "kube-api-access-dwd9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:21:59.195655 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.195557 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4bf15ef-139b-47ed-8944-b24d28f0ea3d" (UID: "d4bf15ef-139b-47ed-8944-b24d28f0ea3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:21:59.295045 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.295015 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwd9m\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-kube-api-access-dwd9m\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.295045 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.295039 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.295045 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.295050 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.295255 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.295057 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4bf15ef-139b-47ed-8944-b24d28f0ea3d-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:21:59.594461 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.594420 2558 generic.go:356] "Generic (PLEG): container finished" podID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerID="3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418" exitCode=0 Sep 07 09:21:59.594601 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.594466 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" Sep 07 09:21:59.594601 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.594505 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" event={"ID":"d4bf15ef-139b-47ed-8944-b24d28f0ea3d","Type":"ContainerDied","Data":"3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418"} Sep 07 09:21:59.594601 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.594561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w" event={"ID":"d4bf15ef-139b-47ed-8944-b24d28f0ea3d","Type":"ContainerDied","Data":"5a445f0933150c75b758ea8b078cd88c0241d37717030b774efa71d061bd4400"} Sep 07 09:21:59.594601 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.594583 2558 scope.go:117] "RemoveContainer" containerID="3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418" Sep 07 09:21:59.604788 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.604772 2558 scope.go:117] "RemoveContainer" containerID="c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177" Sep 07 09:21:59.613759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.613581 2558 scope.go:117] "RemoveContainer" containerID="3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418" Sep 07 09:21:59.613887 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:21:59.613868 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418\": container with ID starting with 3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418 not found: ID does not exist" containerID="3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418" Sep 07 09:21:59.613943 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.613896 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418"} err="failed to get container status \"3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418\": rpc error: code = NotFound desc = could not find container \"3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418\": container with ID starting with 3844d94b200acd990f0c91a6dad05afecdce7ca3081f00fac32b6567024e8418 not found: ID does not exist" Sep 07 09:21:59.613943 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.613917 2558 scope.go:117] "RemoveContainer" containerID="c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177" Sep 07 09:21:59.614182 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:21:59.614163 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177\": container with ID starting with c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177 not found: ID does not exist" containerID="c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177" Sep 07 09:21:59.614268 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.614191 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177"} err="failed to get container status \"c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177\": rpc error: code = NotFound desc = could not find container \"c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177\": container with ID starting with c47231f85b46d37fd131a885bad0fccc5061c89500fe88c565b76d997252b177 not found: ID does not exist" Sep 07 09:21:59.616032 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.616011 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w"] Sep 07 09:21:59.619680 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.619659 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-12fc0fc8-openshift-default-95dd555fb-z5b8w"] Sep 07 09:21:59.958768 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:21:59.958677 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" path="/var/lib/kubelet/pods/d4bf15ef-139b-47ed-8944-b24d28f0ea3d/volumes" Sep 07 09:22:02.123259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:02.123228 2558 scope.go:117] "RemoveContainer" containerID="8e6e26f6dfd323018043472b6bb37592ade2d0d832da0e8224782a43e6a4366f" Sep 07 09:22:02.134689 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:02.134666 2558 scope.go:117] "RemoveContainer" containerID="0a08ce4fade2545c5ecf3880582c5b53726beda83d27aba57fab61ee39190942" Sep 07 09:22:48.135261 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135230 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd"] Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135660 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5cc722-15b5-4f62-989a-6295798edbae" containerName="payload-pre-processing" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135675 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5cc722-15b5-4f62-989a-6295798edbae" containerName="payload-pre-processing" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135686 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874" containerName="revoke-keys" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135695 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874" containerName="revoke-keys" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135709 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135715 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135723 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135730 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135740 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b32765e-0d73-40ff-be76-07d589cc5f77" containerName="main" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135748 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b32765e-0d73-40ff-be76-07d589cc5f77" containerName="main" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135810 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b32765e-0d73-40ff-be76-07d589cc5f77" containerName="main" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135817 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135824 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6cc3743-1f2e-48e2-bd4a-ea8dcb3a9874" containerName="revoke-keys" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135830 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4bf15ef-139b-47ed-8944-b24d28f0ea3d" containerName="istio-proxy" Sep 07 09:22:48.136111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.135837 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5cc722-15b5-4f62-989a-6295798edbae" containerName="payload-pre-processing" Sep 07 09:22:48.139080 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.139062 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.141503 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.141482 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ce194a-openshift-default-dockercfg-7c87j\"" Sep 07 09:22:48.149442 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.149421 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd"] Sep 07 09:22:48.162601 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162578 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-credential-socket\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.162707 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162620 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94bz\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-kube-api-access-d94bz\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.162707 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162671 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-socket\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.162829 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162707 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-token\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.162829 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162783 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-envoy\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.162935 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162843 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-podinfo\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.162935 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162888 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-data\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.163040 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162942 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6e8ab3-c705-42c2-a76b-435a3023e217-istiod-ca-cert\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.163040 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.162977 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-certs\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264035 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.263998 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-podinfo\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264035 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264041 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-data\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264314 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6e8ab3-c705-42c2-a76b-435a3023e217-istiod-ca-cert\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264314 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264208 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-certs\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264314 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264246 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-credential-socket\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264314 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d94bz\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-kube-api-access-d94bz\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264431 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-data\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264473 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264445 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-socket\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264617 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264503 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-token\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264617 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264560 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-envoy\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264769 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-credential-socket\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264838 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264754 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-socket\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.264911 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264888 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-certs\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.265054 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.264950 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6e8ab3-c705-42c2-a76b-435a3023e217-istiod-ca-cert\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.266632 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.266612 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-podinfo\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.266787 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.266769 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-envoy\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.272563 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.272543 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-token\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.272727 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.272706 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94bz\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-kube-api-access-d94bz\") pod \"e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.449877 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.449793 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:48.726458 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.726429 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd"] Sep 07 09:22:48.727489 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:22:48.727459 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6e8ab3_c705_42c2_a76b_435a3023e217.slice/crio-4329d1213113d61775408260576452d0797166ecfebcb9fc66ea7b79dfe9d9ae WatchSource:0}: Error finding container 4329d1213113d61775408260576452d0797166ecfebcb9fc66ea7b79dfe9d9ae: Status 404 returned error can't find the container with id 4329d1213113d61775408260576452d0797166ecfebcb9fc66ea7b79dfe9d9ae Sep 07 09:22:48.729858 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.729826 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:22:48.729944 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.729890 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:22:48.729944 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.729919 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:22:48.895214 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.895178 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" event={"ID":"8c6e8ab3-c705-42c2-a76b-435a3023e217","Type":"ContainerStarted","Data":"3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537"} Sep 07 09:22:48.895446 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.895236 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" event={"ID":"8c6e8ab3-c705-42c2-a76b-435a3023e217","Type":"ContainerStarted","Data":"4329d1213113d61775408260576452d0797166ecfebcb9fc66ea7b79dfe9d9ae"} Sep 07 09:22:48.915018 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:48.914962 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podStartSLOduration=0.91494641 podStartE2EDuration="914.94641ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:22:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:22:48.913043171 +0000 UTC m=+1427.429834077" watchObservedRunningTime="2026-09-07 09:22:48.91494641 +0000 UTC m=+1427.431737312" Sep 07 09:22:49.449959 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:49.449921 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:49.451088 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:49.451059 2558 patch_prober.go:28] interesting pod/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Sep 07 09:22:49.451200 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:49.451112 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Sep 07 09:22:50.454537 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:50.454506 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:50.906177 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:50.906145 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:50.907135 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:50.907117 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:22:52.544135 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.544097 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49"] Sep 07 09:22:52.547997 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.547979 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.550618 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.550596 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ce194a-dockercfg-xtvz5\"" Sep 07 09:22:52.550739 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.550713 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ce194a\"" Sep 07 09:22:52.557721 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.557694 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49"] Sep 07 09:22:52.584910 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.584884 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c"] Sep 07 09:22:52.588343 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.588326 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:52.596460 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.596439 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c"] Sep 07 09:22:52.605833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.605810 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4kdj\" (UniqueName: \"kubernetes.io/projected/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-kube-api-access-n4kdj\") pod \"payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.605936 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.605858 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.605936 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.605887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58011d1c-9309-4185-821e-2e50a2b5db10-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:52.606056 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.606029 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkf4c\" (UniqueName: \"kubernetes.io/projected/58011d1c-9309-4185-821e-2e50a2b5db10-kube-api-access-mkf4c\") pod \"payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:52.707012 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.706976 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.707212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.707029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58011d1c-9309-4185-821e-2e50a2b5db10-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:52.707212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.707082 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkf4c\" (UniqueName: \"kubernetes.io/projected/58011d1c-9309-4185-821e-2e50a2b5db10-kube-api-access-mkf4c\") pod \"payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:52.707212 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.707128 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4kdj\" (UniqueName: \"kubernetes.io/projected/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-kube-api-access-n4kdj\") pod \"payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.707770 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.707746 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58011d1c-9309-4185-821e-2e50a2b5db10-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:52.707864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.707777 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.717373 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.717353 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkf4c\" (UniqueName: \"kubernetes.io/projected/58011d1c-9309-4185-821e-2e50a2b5db10-kube-api-access-mkf4c\") pod \"payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:52.717536 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.717516 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4kdj\" (UniqueName: \"kubernetes.io/projected/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-kube-api-access-n4kdj\") pod \"payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.860096 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.860016 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:52.899359 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:52.899330 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:53.133178 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.133151 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49"] Sep 07 09:22:53.134806 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:22:53.134767 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a34ec75_27ee_45b8_bcb6_0fefbeef113f.slice/crio-e93ab14413b848f2526ad7a019aa5d9f4bef4f2edaf1ff2c526a52053efe6e45 WatchSource:0}: Error finding container e93ab14413b848f2526ad7a019aa5d9f4bef4f2edaf1ff2c526a52053efe6e45: Status 404 returned error can't find the container with id e93ab14413b848f2526ad7a019aa5d9f4bef4f2edaf1ff2c526a52053efe6e45 Sep 07 09:22:53.184152 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.184130 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c"] Sep 07 09:22:53.184846 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:22:53.184829 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58011d1c_9309_4185_821e_2e50a2b5db10.slice/crio-0265b1506226cd2b63cf738712ed30e356e56b6ed541300ef05cc3fae94ac8ff WatchSource:0}: Error finding container 0265b1506226cd2b63cf738712ed30e356e56b6ed541300ef05cc3fae94ac8ff: Status 404 returned error can't find the container with id 0265b1506226cd2b63cf738712ed30e356e56b6ed541300ef05cc3fae94ac8ff Sep 07 09:22:53.931764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.931724 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" event={"ID":"58011d1c-9309-4185-821e-2e50a2b5db10","Type":"ContainerStarted","Data":"2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8"} Sep 07 09:22:53.932222 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.931890 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:22:53.932222 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.931937 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" event={"ID":"58011d1c-9309-4185-821e-2e50a2b5db10","Type":"ContainerStarted","Data":"0265b1506226cd2b63cf738712ed30e356e56b6ed541300ef05cc3fae94ac8ff"} Sep 07 09:22:53.933424 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.933402 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" event={"ID":"1a34ec75-27ee-45b8-bcb6-0fefbeef113f","Type":"ContainerStarted","Data":"2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd"} Sep 07 09:22:53.933520 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.933438 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" event={"ID":"1a34ec75-27ee-45b8-bcb6-0fefbeef113f","Type":"ContainerStarted","Data":"e93ab14413b848f2526ad7a019aa5d9f4bef4f2edaf1ff2c526a52053efe6e45"} Sep 07 09:22:53.933579 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.933523 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:22:53.947960 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.947909 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" podStartSLOduration=1.947892574 podStartE2EDuration="1.947892574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:22:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:22:53.947107919 +0000 UTC m=+1432.463898823" watchObservedRunningTime="2026-09-07 09:22:53.947892574 +0000 UTC m=+1432.464683477" Sep 07 09:22:53.966790 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:22:53.966741 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" podStartSLOduration=1.966725454 podStartE2EDuration="1.966725454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:22:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:22:53.963502496 +0000 UTC m=+1432.480293400" watchObservedRunningTime="2026-09-07 09:22:53.966725454 +0000 UTC m=+1432.483516358" Sep 07 09:23:02.187489 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:02.187416 2558 scope.go:117] "RemoveContainer" containerID="8b60f21087ae64d172745c141c74ab13fbb9c4dd49e86a09ecf9c3d274d2c988" Sep 07 09:23:04.942042 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:04.941995 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:23:04.943183 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:04.943158 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:23:14.067262 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:14.067221 2558 generic.go:356] "Generic (PLEG): container finished" podID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerID="3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537" exitCode=0 Sep 07 09:23:14.067689 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:14.067312 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" event={"ID":"8c6e8ab3-c705-42c2-a76b-435a3023e217","Type":"ContainerDied","Data":"3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537"} Sep 07 09:23:14.067737 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:14.067690 2558 scope.go:117] "RemoveContainer" containerID="3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537" Sep 07 09:23:14.068798 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:14.068772 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:14.068852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:14.068839 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:14.068889 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:14.068872 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:15.075157 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:15.075122 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" event={"ID":"8c6e8ab3-c705-42c2-a76b-435a3023e217","Type":"ContainerStarted","Data":"fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b"} Sep 07 09:23:15.450252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:15.450153 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:23:15.450435 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:15.450385 2558 patch_prober.go:28] interesting pod/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Sep 07 09:23:15.450482 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:15.450433 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Sep 07 09:23:16.437761 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.437720 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8"] Sep 07 09:23:16.451249 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.451213 2558 patch_prober.go:28] interesting pod/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Sep 07 09:23:16.451519 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.451487 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Sep 07 09:23:16.452550 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.452526 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.454731 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.452979 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8"] Sep 07 09:23:16.455631 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.455607 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-558343-openshift-default-dockercfg-kf7sw\"" Sep 07 09:23:16.552155 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552097 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-socket\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552385 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552169 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxmq\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-kube-api-access-2jxmq\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552385 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552234 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-token\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552385 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552284 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istiod-ca-cert\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552385 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552371 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-envoy\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552621 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552422 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-podinfo\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552621 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552504 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-data\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552621 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552527 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-certs\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.552621 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.552560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-credential-socket\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.653765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.653727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-data\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.653765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.653764 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-certs\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.653788 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-credential-socket\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.653818 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-socket\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.653844 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxmq\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-kube-api-access-2jxmq\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.653885 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-token\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654024 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.653919 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istiod-ca-cert\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654278 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.654145 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-envoy\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654278 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.654196 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-certs\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654278 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.654213 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-podinfo\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654449 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.654333 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-credential-socket\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654449 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.654425 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-socket\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.654490 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-data\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.654790 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.654770 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istiod-ca-cert\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.656406 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.656385 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-envoy\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.656560 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.656543 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-podinfo\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.661618 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.661596 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxmq\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-kube-api-access-2jxmq\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.661791 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.661772 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-token\") pod \"e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:16.768858 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:16.768754 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:17.069957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:17.069934 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8"] Sep 07 09:23:17.071236 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:23:17.071211 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c0cafbc_3e23_4a44_89f9_fd1afa0f7c49.slice/crio-ae761e99c946302b65c977a6d9dac4923eead7d5afc52794134b66442cf150e3 WatchSource:0}: Error finding container ae761e99c946302b65c977a6d9dac4923eead7d5afc52794134b66442cf150e3: Status 404 returned error can't find the container with id ae761e99c946302b65c977a6d9dac4923eead7d5afc52794134b66442cf150e3 Sep 07 09:23:17.073587 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:17.073557 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:17.073657 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:17.073619 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:17.073657 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:17.073645 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:17.088365 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:17.088336 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" event={"ID":"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49","Type":"ContainerStarted","Data":"ae761e99c946302b65c977a6d9dac4923eead7d5afc52794134b66442cf150e3"} Sep 07 09:23:17.450817 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:17.450734 2558 patch_prober.go:28] interesting pod/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Sep 07 09:23:17.450817 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:17.450791 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Sep 07 09:23:18.095475 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:18.095438 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" event={"ID":"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49","Type":"ContainerStarted","Data":"5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d"} Sep 07 09:23:18.114526 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:18.114478 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podStartSLOduration=2.114463096 podStartE2EDuration="2.114463096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:23:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:23:18.11289702 +0000 UTC m=+1456.629687933" watchObservedRunningTime="2026-09-07 09:23:18.114463096 +0000 UTC m=+1456.631253999" Sep 07 09:23:18.451267 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:18.451176 2558 patch_prober.go:28] interesting pod/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Sep 07 09:23:18.451267 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:18.451234 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Sep 07 09:23:18.768971 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:18.768891 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:18.774356 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:18.774331 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:19.101954 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:19.101919 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:19.102961 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:19.102937 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:19.450462 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:19.450373 2558 patch_prober.go:28] interesting pod/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Sep 07 09:23:19.450462 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:19.450435 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Sep 07 09:23:20.454561 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:20.454529 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:23:20.455016 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:20.454922 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:23:20.455610 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:20.455591 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:23:35.144515 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:35.144480 2558 patch_prober.go:28] interesting pod/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 07 09:23:35.144877 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:35.144534 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 07 09:23:37.209846 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:37.209742 2558 generic.go:356] "Generic (PLEG): container finished" podID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerID="5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d" exitCode=0 Sep 07 09:23:37.209846 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:37.209810 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" event={"ID":"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49","Type":"ContainerDied","Data":"5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d"} Sep 07 09:23:37.210279 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:37.210176 2558 scope.go:117] "RemoveContainer" containerID="5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d" Sep 07 09:23:37.211293 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:37.211264 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:37.211385 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:37.211356 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:37.211421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:37.211391 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"} Sep 07 09:23:38.225690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:38.225641 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" event={"ID":"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49","Type":"ContainerStarted","Data":"ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f"} Sep 07 09:23:38.769886 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:38.769851 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:38.770080 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:38.770060 2558 patch_prober.go:28] interesting pod/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 07 09:23:38.770133 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:38.770106 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 07 09:23:39.769609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:39.769575 2558 patch_prober.go:28] interesting pod/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 07 09:23:39.770012 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:39.769624 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 07 09:23:40.769350 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:40.769291 2558 patch_prober.go:28] interesting pod/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 07 09:23:40.769532 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:40.769379 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 07 09:23:41.769621 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:41.769587 2558 patch_prober.go:28] interesting pod/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 07 09:23:41.770090 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:41.769658 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 07 09:23:42.769143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:42.769105 2558 patch_prober.go:28] interesting pod/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 07 09:23:42.769350 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:42.769160 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 07 09:23:43.773191 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:43.773158 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:43.773640 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:43.773550 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:43.774179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:43.774160 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:23:47.687275 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.687243 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw"] Sep 07 09:23:47.690740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.690724 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" Sep 07 09:23:47.694149 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.694119 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:23:47.694324 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.694251 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:23:47.700083 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.700058 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw"] Sep 07 09:23:47.783222 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.783185 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572eefc6-4668-459e-8728-e8e7887b7cb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce194a-w25nw\" (UID: \"572eefc6-4668-459e-8728-e8e7887b7cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" Sep 07 09:23:47.885468 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.884436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572eefc6-4668-459e-8728-e8e7887b7cb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce194a-w25nw\" (UID: \"572eefc6-4668-459e-8728-e8e7887b7cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" Sep 07 09:23:47.885792 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:47.885423 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572eefc6-4668-459e-8728-e8e7887b7cb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ce194a-w25nw\" (UID: \"572eefc6-4668-459e-8728-e8e7887b7cb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" Sep 07 09:23:48.013193 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:48.013114 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" Sep 07 09:23:48.284360 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:48.284336 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw"] Sep 07 09:23:48.286838 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:23:48.286801 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572eefc6_4668_459e_8728_e8e7887b7cb2.slice/crio-8adaef74fbf04d1254e0326af91a77009d0289abf85829b085e4eb500117dc77 WatchSource:0}: Error finding container 8adaef74fbf04d1254e0326af91a77009d0289abf85829b085e4eb500117dc77: Status 404 returned error can't find the container with id 8adaef74fbf04d1254e0326af91a77009d0289abf85829b085e4eb500117dc77 Sep 07 09:23:49.300458 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:49.300424 2558 generic.go:356] "Generic (PLEG): container finished" podID="572eefc6-4668-459e-8728-e8e7887b7cb2" containerID="d63eaf5eee29392499dca1aa7165c69378de43c731e4ac4fadd52d67573f7a34" exitCode=0 Sep 07 09:23:49.300822 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:49.300505 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" event={"ID":"572eefc6-4668-459e-8728-e8e7887b7cb2","Type":"ContainerDied","Data":"d63eaf5eee29392499dca1aa7165c69378de43c731e4ac4fadd52d67573f7a34"} Sep 07 09:23:49.300822 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:49.300554 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" event={"ID":"572eefc6-4668-459e-8728-e8e7887b7cb2","Type":"ContainerStarted","Data":"8adaef74fbf04d1254e0326af91a77009d0289abf85829b085e4eb500117dc77"} Sep 07 09:23:50.444830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:50.444806 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" Sep 07 09:23:50.512480 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:50.512444 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572eefc6-4668-459e-8728-e8e7887b7cb2-maas-api-service-ca\") pod \"572eefc6-4668-459e-8728-e8e7887b7cb2\" (UID: \"572eefc6-4668-459e-8728-e8e7887b7cb2\") " Sep 07 09:23:50.512847 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:50.512820 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572eefc6-4668-459e-8728-e8e7887b7cb2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "572eefc6-4668-459e-8728-e8e7887b7cb2" (UID: "572eefc6-4668-459e-8728-e8e7887b7cb2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:23:50.512920 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:50.512905 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/572eefc6-4668-459e-8728-e8e7887b7cb2-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:23:51.313509 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:51.313481 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" Sep 07 09:23:51.313682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:51.313518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw" event={"ID":"572eefc6-4668-459e-8728-e8e7887b7cb2","Type":"ContainerDied","Data":"8adaef74fbf04d1254e0326af91a77009d0289abf85829b085e4eb500117dc77"} Sep 07 09:23:51.313682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:23:51.313569 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8adaef74fbf04d1254e0326af91a77009d0289abf85829b085e4eb500117dc77" Sep 07 09:24:02.041205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:02.041177 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:24:02.042948 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:02.042927 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:24:02.208983 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:02.208950 2558 scope.go:117] "RemoveContainer" containerID="bfb42d44f898bd4ce102b4b8778fb8bb596e1fbdc30776784116658b2491f2b1" Sep 07 09:24:02.217701 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:02.217680 2558 scope.go:117] "RemoveContainer" containerID="7c5660ea002314eb61be1dc81b62eba0110d5255b86af35e4973e9cdf68a6a10" Sep 07 09:24:03.437358 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.437325 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c"] Sep 07 09:24:03.437825 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.437603 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" podUID="58011d1c-9309-4185-821e-2e50a2b5db10" containerName="payload-processing" containerID="cri-o://2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8" gracePeriod=30 Sep 07 09:24:03.446111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.444412 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49"] Sep 07 09:24:03.446111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.444658 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" podUID="1a34ec75-27ee-45b8-bcb6-0fefbeef113f" containerName="payload-pre-processing" containerID="cri-o://2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd" gracePeriod=30 Sep 07 09:24:03.585868 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.585837 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw"] Sep 07 09:24:03.590233 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.590202 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ce194a-w25nw"] Sep 07 09:24:03.746549 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.746524 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:24:03.750002 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.749979 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:24:03.845979 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.845947 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-plugins-config-volume\") pod \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " Sep 07 09:24:03.846148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.845999 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkf4c\" (UniqueName: \"kubernetes.io/projected/58011d1c-9309-4185-821e-2e50a2b5db10-kube-api-access-mkf4c\") pod \"58011d1c-9309-4185-821e-2e50a2b5db10\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " Sep 07 09:24:03.846148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.846114 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4kdj\" (UniqueName: \"kubernetes.io/projected/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-kube-api-access-n4kdj\") pod \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\" (UID: \"1a34ec75-27ee-45b8-bcb6-0fefbeef113f\") " Sep 07 09:24:03.846148 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.846132 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58011d1c-9309-4185-821e-2e50a2b5db10-plugins-config-volume\") pod \"58011d1c-9309-4185-821e-2e50a2b5db10\" (UID: \"58011d1c-9309-4185-821e-2e50a2b5db10\") " Sep 07 09:24:03.846483 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.846420 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a34ec75-27ee-45b8-bcb6-0fefbeef113f" (UID: "1a34ec75-27ee-45b8-bcb6-0fefbeef113f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:24:03.846609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.846589 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58011d1c-9309-4185-821e-2e50a2b5db10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58011d1c-9309-4185-821e-2e50a2b5db10" (UID: "58011d1c-9309-4185-821e-2e50a2b5db10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:24:03.848230 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.848209 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58011d1c-9309-4185-821e-2e50a2b5db10-kube-api-access-mkf4c" (OuterVolumeSpecName: "kube-api-access-mkf4c") pod "58011d1c-9309-4185-821e-2e50a2b5db10" (UID: "58011d1c-9309-4185-821e-2e50a2b5db10"). InnerVolumeSpecName "kube-api-access-mkf4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:24:03.848340 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.848322 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-kube-api-access-n4kdj" (OuterVolumeSpecName: "kube-api-access-n4kdj") pod "1a34ec75-27ee-45b8-bcb6-0fefbeef113f" (UID: "1a34ec75-27ee-45b8-bcb6-0fefbeef113f"). InnerVolumeSpecName "kube-api-access-n4kdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:24:03.946732 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.946691 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:03.946732 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.946720 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkf4c\" (UniqueName: \"kubernetes.io/projected/58011d1c-9309-4185-821e-2e50a2b5db10-kube-api-access-mkf4c\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:03.946732 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.946732 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4kdj\" (UniqueName: \"kubernetes.io/projected/1a34ec75-27ee-45b8-bcb6-0fefbeef113f-kube-api-access-n4kdj\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:03.946732 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.946744 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58011d1c-9309-4185-821e-2e50a2b5db10-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:03.953107 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:03.953075 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="572eefc6-4668-459e-8728-e8e7887b7cb2" path="/var/lib/kubelet/pods/572eefc6-4668-459e-8728-e8e7887b7cb2/volumes" Sep 07 09:24:04.401167 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.401130 2558 generic.go:356] "Generic (PLEG): container finished" podID="1a34ec75-27ee-45b8-bcb6-0fefbeef113f" containerID="2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd" exitCode=0 Sep 07 09:24:04.401367 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.401187 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" Sep 07 09:24:04.401367 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.401220 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" event={"ID":"1a34ec75-27ee-45b8-bcb6-0fefbeef113f","Type":"ContainerDied","Data":"2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd"} Sep 07 09:24:04.401367 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.401271 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49" event={"ID":"1a34ec75-27ee-45b8-bcb6-0fefbeef113f","Type":"ContainerDied","Data":"e93ab14413b848f2526ad7a019aa5d9f4bef4f2edaf1ff2c526a52053efe6e45"} Sep 07 09:24:04.401367 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.401313 2558 scope.go:117] "RemoveContainer" containerID="2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd" Sep 07 09:24:04.402884 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.402858 2558 generic.go:356] "Generic (PLEG): container finished" podID="58011d1c-9309-4185-821e-2e50a2b5db10" containerID="2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8" exitCode=0 Sep 07 09:24:04.403002 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.402897 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" Sep 07 09:24:04.403002 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.402945 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" event={"ID":"58011d1c-9309-4185-821e-2e50a2b5db10","Type":"ContainerDied","Data":"2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8"} Sep 07 09:24:04.403129 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.403001 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c" event={"ID":"58011d1c-9309-4185-821e-2e50a2b5db10","Type":"ContainerDied","Data":"0265b1506226cd2b63cf738712ed30e356e56b6ed541300ef05cc3fae94ac8ff"} Sep 07 09:24:04.411610 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.411598 2558 scope.go:117] "RemoveContainer" containerID="2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd" Sep 07 09:24:04.411901 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:24:04.411862 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd\": container with ID starting with 2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd not found: ID does not exist" containerID="2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd" Sep 07 09:24:04.411993 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.411907 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd"} err="failed to get container status \"2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd\": rpc error: code = NotFound desc = could not find container \"2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd\": container with ID starting with 2bd8a6d921ca3e13676a4191d8921e78f218ed56240ae363c409a62231b886fd not found: ID does not exist" Sep 07 09:24:04.411993 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.411923 2558 scope.go:117] "RemoveContainer" containerID="2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8" Sep 07 09:24:04.420599 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.420571 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c"] Sep 07 09:24:04.423911 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.423609 2558 scope.go:117] "RemoveContainer" containerID="2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8" Sep 07 09:24:04.424092 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:24:04.424067 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8\": container with ID starting with 2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8 not found: ID does not exist" containerID="2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8" Sep 07 09:24:04.424159 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.424102 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8"} err="failed to get container status \"2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8\": rpc error: code = NotFound desc = could not find container \"2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8\": container with ID starting with 2a82ca89c95439c11cab28a66996f8e8ef54e63b8b550fdb7ecf5d7b5cce7dc8 not found: ID does not exist" Sep 07 09:24:04.425323 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.425275 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ce194a-5c4f8f645-lsl9c"] Sep 07 09:24:04.433292 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.433272 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49"] Sep 07 09:24:04.438736 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:04.438718 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ce194a-7d5c68c746-pxd49"] Sep 07 09:24:05.951443 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:05.951409 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a34ec75-27ee-45b8-bcb6-0fefbeef113f" path="/var/lib/kubelet/pods/1a34ec75-27ee-45b8-bcb6-0fefbeef113f/volumes" Sep 07 09:24:05.951837 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:05.951772 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58011d1c-9309-4185-821e-2e50a2b5db10" path="/var/lib/kubelet/pods/58011d1c-9309-4185-821e-2e50a2b5db10/volumes" Sep 07 09:24:09.941768 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:09.941737 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd"] Sep 07 09:24:09.942159 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:09.941945 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" containerID="cri-o://fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b" gracePeriod=30 Sep 07 09:24:10.296889 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.296847 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5"] Sep 07 09:24:10.297812 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.297786 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="572eefc6-4668-459e-8728-e8e7887b7cb2" containerName="revoke-keys" Sep 07 09:24:10.297812 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.297813 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="572eefc6-4668-459e-8728-e8e7887b7cb2" containerName="revoke-keys" Sep 07 09:24:10.298004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.297841 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a34ec75-27ee-45b8-bcb6-0fefbeef113f" containerName="payload-pre-processing" Sep 07 09:24:10.298004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.297851 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a34ec75-27ee-45b8-bcb6-0fefbeef113f" containerName="payload-pre-processing" Sep 07 09:24:10.298004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.297870 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58011d1c-9309-4185-821e-2e50a2b5db10" containerName="payload-processing" Sep 07 09:24:10.298004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.297879 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="58011d1c-9309-4185-821e-2e50a2b5db10" containerName="payload-processing" Sep 07 09:24:10.298187 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.298080 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="58011d1c-9309-4185-821e-2e50a2b5db10" containerName="payload-processing" Sep 07 09:24:10.298187 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.298100 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a34ec75-27ee-45b8-bcb6-0fefbeef113f" containerName="payload-pre-processing" Sep 07 09:24:10.298187 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.298118 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="572eefc6-4668-459e-8728-e8e7887b7cb2" containerName="revoke-keys" Sep 07 09:24:10.303942 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.303915 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" Sep 07 09:24:10.304832 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.304810 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5"] Sep 07 09:24:10.306336 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.306313 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:24:10.306472 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.306323 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:24:10.409832 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.409791 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a325096-e928-4cff-9104-7e820c0d046a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-558343-sdlw5\" (UID: \"3a325096-e928-4cff-9104-7e820c0d046a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" Sep 07 09:24:10.510478 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.510447 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a325096-e928-4cff-9104-7e820c0d046a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-558343-sdlw5\" (UID: \"3a325096-e928-4cff-9104-7e820c0d046a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" Sep 07 09:24:10.511082 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.511062 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a325096-e928-4cff-9104-7e820c0d046a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-558343-sdlw5\" (UID: \"3a325096-e928-4cff-9104-7e820c0d046a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" Sep 07 09:24:10.617247 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.617163 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" Sep 07 09:24:10.888856 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:10.888824 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5"] Sep 07 09:24:10.889587 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:24:10.889560 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a325096_e928_4cff_9104_7e820c0d046a.slice/crio-635a4864111e8f339045c378ac9fd5ea93994ab2833f7703bf59b9b335705fd7 WatchSource:0}: Error finding container 635a4864111e8f339045c378ac9fd5ea93994ab2833f7703bf59b9b335705fd7: Status 404 returned error can't find the container with id 635a4864111e8f339045c378ac9fd5ea93994ab2833f7703bf59b9b335705fd7 Sep 07 09:24:11.454245 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:11.454211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" event={"ID":"3a325096-e928-4cff-9104-7e820c0d046a","Type":"ContainerStarted","Data":"c47a1305ba3968812fcca2e1f3bfbff52b928bbd180de876ae555c0262702416"} Sep 07 09:24:11.454649 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:11.454262 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" event={"ID":"3a325096-e928-4cff-9104-7e820c0d046a","Type":"ContainerStarted","Data":"635a4864111e8f339045c378ac9fd5ea93994ab2833f7703bf59b9b335705fd7"} Sep 07 09:24:11.470826 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:11.470779 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" podStartSLOduration=1.470763527 podStartE2EDuration="1.470763527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:24:11.467984709 +0000 UTC m=+1509.984775613" watchObservedRunningTime="2026-09-07 09:24:11.470763527 +0000 UTC m=+1509.987554430" Sep 07 09:24:12.461373 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:12.461337 2558 generic.go:356] "Generic (PLEG): container finished" podID="3a325096-e928-4cff-9104-7e820c0d046a" containerID="c47a1305ba3968812fcca2e1f3bfbff52b928bbd180de876ae555c0262702416" exitCode=0 Sep 07 09:24:12.461779 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:12.461390 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" event={"ID":"3a325096-e928-4cff-9104-7e820c0d046a","Type":"ContainerDied","Data":"c47a1305ba3968812fcca2e1f3bfbff52b928bbd180de876ae555c0262702416"} Sep 07 09:24:13.601623 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:13.601601 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" Sep 07 09:24:13.640234 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:13.640205 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a325096-e928-4cff-9104-7e820c0d046a-maas-api-service-ca\") pod \"3a325096-e928-4cff-9104-7e820c0d046a\" (UID: \"3a325096-e928-4cff-9104-7e820c0d046a\") " Sep 07 09:24:13.640549 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:13.640526 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a325096-e928-4cff-9104-7e820c0d046a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3a325096-e928-4cff-9104-7e820c0d046a" (UID: "3a325096-e928-4cff-9104-7e820c0d046a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:24:13.741889 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:13.741803 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a325096-e928-4cff-9104-7e820c0d046a-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:14.474567 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:14.474541 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" Sep 07 09:24:14.474567 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:14.474556 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5" event={"ID":"3a325096-e928-4cff-9104-7e820c0d046a","Type":"ContainerDied","Data":"635a4864111e8f339045c378ac9fd5ea93994ab2833f7703bf59b9b335705fd7"} Sep 07 09:24:14.474780 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:14.474600 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="635a4864111e8f339045c378ac9fd5ea93994ab2833f7703bf59b9b335705fd7" Sep 07 09:24:15.222725 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.222701 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:24:15.256602 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256574 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-socket\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.256761 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256628 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d94bz\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-kube-api-access-d94bz\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.256761 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256682 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-envoy\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.256885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256777 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-podinfo\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.256885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256825 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-certs\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.256998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256877 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:15.256998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256887 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6e8ab3-c705-42c2-a76b-435a3023e217-istiod-ca-cert\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.256998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256946 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-credential-socket\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.256998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.256981 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-data\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.257202 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257030 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-token\") pod \"8c6e8ab3-c705-42c2-a76b-435a3023e217\" (UID: \"8c6e8ab3-c705-42c2-a76b-435a3023e217\") " Sep 07 09:24:15.257264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257192 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:15.257264 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257234 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c6e8ab3-c705-42c2-a76b-435a3023e217-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:24:15.257400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257254 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:15.257400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257384 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:15.257523 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257502 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c6e8ab3-c705-42c2-a76b-435a3023e217-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.257582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257529 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.257582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257544 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.257582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257558 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.257582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.257569 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.258975 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.258951 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:24:15.259221 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.259205 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:24:15.259364 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.259327 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-kube-api-access-d94bz" (OuterVolumeSpecName: "kube-api-access-d94bz") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "kube-api-access-d94bz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:24:15.259505 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.259489 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c6e8ab3-c705-42c2-a76b-435a3023e217" (UID: "8c6e8ab3-c705-42c2-a76b-435a3023e217"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:15.358311 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.358274 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d94bz\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-kube-api-access-d94bz\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.358490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.358327 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.358490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.358337 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.358490 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.358345 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c6e8ab3-c705-42c2-a76b-435a3023e217-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:15.482594 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.482495 2558 generic.go:356] "Generic (PLEG): container finished" podID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerID="fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b" exitCode=0 Sep 07 09:24:15.482594 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.482541 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" Sep 07 09:24:15.482844 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.482590 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" event={"ID":"8c6e8ab3-c705-42c2-a76b-435a3023e217","Type":"ContainerDied","Data":"fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b"} Sep 07 09:24:15.482844 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.482647 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd" event={"ID":"8c6e8ab3-c705-42c2-a76b-435a3023e217","Type":"ContainerDied","Data":"4329d1213113d61775408260576452d0797166ecfebcb9fc66ea7b79dfe9d9ae"} Sep 07 09:24:15.482844 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.482678 2558 scope.go:117] "RemoveContainer" containerID="fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b" Sep 07 09:24:15.492680 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.492656 2558 scope.go:117] "RemoveContainer" containerID="3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537" Sep 07 09:24:15.501030 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.501010 2558 scope.go:117] "RemoveContainer" containerID="fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b" Sep 07 09:24:15.501673 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:24:15.501354 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b\": container with ID starting with fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b not found: ID does not exist" containerID="fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b" Sep 07 09:24:15.501673 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.501391 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b"} err="failed to get container status \"fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b\": rpc error: code = NotFound desc = could not find container \"fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b\": container with ID starting with fa0c49f9c34cb3f6b5674a6c8fbda54a7a4eab0e1f4b46bf82f7036ce628293b not found: ID does not exist" Sep 07 09:24:15.501673 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.501443 2558 scope.go:117] "RemoveContainer" containerID="3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537" Sep 07 09:24:15.501866 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:24:15.501705 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537\": container with ID starting with 3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537 not found: ID does not exist" containerID="3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537" Sep 07 09:24:15.501866 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.501730 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537"} err="failed to get container status \"3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537\": rpc error: code = NotFound desc = could not find container \"3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537\": container with ID starting with 3716eb72f7a19990147ad255713d30937d7f2ef9e11e2746160454ccbc77f537 not found: ID does not exist" Sep 07 09:24:15.504390 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.504372 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd"] Sep 07 09:24:15.507997 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.507978 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ce194a-openshift-default-7bd484597d-cslhd"] Sep 07 09:24:15.951770 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:15.951734 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" path="/var/lib/kubelet/pods/8c6e8ab3-c705-42c2-a76b-435a3023e217/volumes" Sep 07 09:24:25.843093 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:25.842996 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5"] Sep 07 09:24:25.845609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:25.845582 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-558343-sdlw5"] Sep 07 09:24:25.956125 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:25.956084 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a325096-e928-4cff-9104-7e820c0d046a" path="/var/lib/kubelet/pods/3a325096-e928-4cff-9104-7e820c0d046a/volumes" Sep 07 09:24:32.159512 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.159480 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8"] Sep 07 09:24:32.160109 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.159711 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" containerID="cri-o://ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f" gracePeriod=30 Sep 07 09:24:32.474266 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474191 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l"] Sep 07 09:24:32.474626 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474612 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" Sep 07 09:24:32.474671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474628 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" Sep 07 09:24:32.474671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474643 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a325096-e928-4cff-9104-7e820c0d046a" containerName="revoke-keys" Sep 07 09:24:32.474671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474649 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a325096-e928-4cff-9104-7e820c0d046a" containerName="revoke-keys" Sep 07 09:24:32.474772 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474731 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" Sep 07 09:24:32.474772 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474740 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" Sep 07 09:24:32.474772 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.474749 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a325096-e928-4cff-9104-7e820c0d046a" containerName="revoke-keys" Sep 07 09:24:32.479243 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.479227 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" Sep 07 09:24:32.481762 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.481737 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:24:32.481936 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.481919 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:24:32.483924 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.483886 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l"] Sep 07 09:24:32.620714 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.620679 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d37e491-4642-461c-a9bb-54ea0af1ad9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l\" (UID: \"1d37e491-4642-461c-a9bb-54ea0af1ad9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" Sep 07 09:24:32.721693 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.721659 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d37e491-4642-461c-a9bb-54ea0af1ad9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l\" (UID: \"1d37e491-4642-461c-a9bb-54ea0af1ad9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" Sep 07 09:24:32.722258 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.722238 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d37e491-4642-461c-a9bb-54ea0af1ad9d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l\" (UID: \"1d37e491-4642-461c-a9bb-54ea0af1ad9d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" Sep 07 09:24:32.792719 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:32.792696 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" Sep 07 09:24:33.066244 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:33.066218 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l"] Sep 07 09:24:33.067591 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:24:33.067561 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d37e491_4642_461c_a9bb_54ea0af1ad9d.slice/crio-597782412d39d188c4ff6ebd93704944dff5862d65f0e725018c67221b80daee WatchSource:0}: Error finding container 597782412d39d188c4ff6ebd93704944dff5862d65f0e725018c67221b80daee: Status 404 returned error can't find the container with id 597782412d39d188c4ff6ebd93704944dff5862d65f0e725018c67221b80daee Sep 07 09:24:33.602998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:33.602907 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" event={"ID":"1d37e491-4642-461c-a9bb-54ea0af1ad9d","Type":"ContainerStarted","Data":"710030094a53be3abc2e34926c67e1fc08e3482e6d376516e97b43c7e2a06bf3"} Sep 07 09:24:33.602998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:33.602961 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" event={"ID":"1d37e491-4642-461c-a9bb-54ea0af1ad9d","Type":"ContainerStarted","Data":"597782412d39d188c4ff6ebd93704944dff5862d65f0e725018c67221b80daee"} Sep 07 09:24:33.619186 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:33.619139 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" podStartSLOduration=1.619122864 podStartE2EDuration="1.619122864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:24:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:24:33.616672388 +0000 UTC m=+1532.133463291" watchObservedRunningTime="2026-09-07 09:24:33.619122864 +0000 UTC m=+1532.135913767" Sep 07 09:24:34.610155 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:34.610119 2558 generic.go:356] "Generic (PLEG): container finished" podID="1d37e491-4642-461c-a9bb-54ea0af1ad9d" containerID="710030094a53be3abc2e34926c67e1fc08e3482e6d376516e97b43c7e2a06bf3" exitCode=0 Sep 07 09:24:34.610618 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:34.610205 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" event={"ID":"1d37e491-4642-461c-a9bb-54ea0af1ad9d","Type":"ContainerDied","Data":"710030094a53be3abc2e34926c67e1fc08e3482e6d376516e97b43c7e2a06bf3"} Sep 07 09:24:35.743806 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:35.743783 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" Sep 07 09:24:35.850486 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:35.850455 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d37e491-4642-461c-a9bb-54ea0af1ad9d-maas-api-service-ca\") pod \"1d37e491-4642-461c-a9bb-54ea0af1ad9d\" (UID: \"1d37e491-4642-461c-a9bb-54ea0af1ad9d\") " Sep 07 09:24:35.850776 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:35.850755 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d37e491-4642-461c-a9bb-54ea0af1ad9d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1d37e491-4642-461c-a9bb-54ea0af1ad9d" (UID: "1d37e491-4642-461c-a9bb-54ea0af1ad9d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:24:35.952154 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:35.952069 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d37e491-4642-461c-a9bb-54ea0af1ad9d-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:36.623795 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:36.623769 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" Sep 07 09:24:36.623971 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:36.623762 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l" event={"ID":"1d37e491-4642-461c-a9bb-54ea0af1ad9d","Type":"ContainerDied","Data":"597782412d39d188c4ff6ebd93704944dff5862d65f0e725018c67221b80daee"} Sep 07 09:24:36.623971 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:36.623894 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="597782412d39d188c4ff6ebd93704944dff5862d65f0e725018c67221b80daee" Sep 07 09:24:37.416667 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.416638 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:24:37.569085 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569057 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jxmq\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-kube-api-access-2jxmq\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569279 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569121 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-socket\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569279 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569148 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-podinfo\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569279 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569188 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-certs\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569279 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569242 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-token\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569279 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569276 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-envoy\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569323 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-credential-socket\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569373 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istiod-ca-cert\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569412 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-data\") pod \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\" (UID: \"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49\") " Sep 07 09:24:37.569581 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569517 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:37.569792 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569656 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:37.569792 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569716 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:37.569792 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569750 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:24:37.570005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569824 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.570005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569829 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:37.570005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569842 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.570005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569858 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.570005 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.569869 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.571455 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.571432 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:24:37.571570 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.571494 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:24:37.571570 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.571522 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-kube-api-access-2jxmq" (OuterVolumeSpecName: "kube-api-access-2jxmq") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "kube-api-access-2jxmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:24:37.571651 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.571573 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" (UID: "2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:24:37.634582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.634549 2558 generic.go:356] "Generic (PLEG): container finished" podID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerID="ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f" exitCode=0 Sep 07 09:24:37.634740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.634597 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" Sep 07 09:24:37.634740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.634629 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" event={"ID":"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49","Type":"ContainerDied","Data":"ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f"} Sep 07 09:24:37.634740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.634682 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8" event={"ID":"2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49","Type":"ContainerDied","Data":"ae761e99c946302b65c977a6d9dac4923eead7d5afc52794134b66442cf150e3"} Sep 07 09:24:37.634740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.634706 2558 scope.go:117] "RemoveContainer" containerID="ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f" Sep 07 09:24:37.644636 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.644620 2558 scope.go:117] "RemoveContainer" containerID="5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d" Sep 07 09:24:37.653413 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.653386 2558 scope.go:117] "RemoveContainer" containerID="ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f" Sep 07 09:24:37.653673 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:24:37.653653 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f\": container with ID starting with ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f not found: ID does not exist" containerID="ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f" Sep 07 09:24:37.653756 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.653684 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f"} err="failed to get container status \"ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f\": rpc error: code = NotFound desc = could not find container \"ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f\": container with ID starting with ffef2d7ee49779c5d95531590027248938189d6b19157c163288b963bb175f2f not found: ID does not exist" Sep 07 09:24:37.653756 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.653707 2558 scope.go:117] "RemoveContainer" containerID="5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d" Sep 07 09:24:37.654088 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:24:37.654021 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d\": container with ID starting with 5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d not found: ID does not exist" containerID="5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d" Sep 07 09:24:37.654088 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.654052 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d"} err="failed to get container status \"5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d\": rpc error: code = NotFound desc = could not find container \"5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d\": container with ID starting with 5f3c9e1cdff743db8fe85fbb259595a67fee4a0a5f7110677caa6d1a5363584d not found: ID does not exist" Sep 07 09:24:37.657725 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.657704 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8"] Sep 07 09:24:37.661176 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.661151 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-558343-openshift-default-57c7d5465d-c5tg8"] Sep 07 09:24:37.671032 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.671011 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.671032 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.671030 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.671130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.671059 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jxmq\" (UniqueName: \"kubernetes.io/projected/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-kube-api-access-2jxmq\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.671130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.671068 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.671130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.671077 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:37.951728 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:37.951647 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" path="/var/lib/kubelet/pods/2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49/volumes" Sep 07 09:24:43.023332 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:43.023279 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l"] Sep 07 09:24:43.027634 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:43.027609 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-263421-kn25l"] Sep 07 09:24:43.951181 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:43.951141 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d37e491-4642-461c-a9bb-54ea0af1ad9d" path="/var/lib/kubelet/pods/1d37e491-4642-461c-a9bb-54ea0af1ad9d/volumes" Sep 07 09:24:49.573364 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573326 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6"] Sep 07 09:24:49.573784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573736 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d37e491-4642-461c-a9bb-54ea0af1ad9d" containerName="revoke-keys" Sep 07 09:24:49.573784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573748 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d37e491-4642-461c-a9bb-54ea0af1ad9d" containerName="revoke-keys" Sep 07 09:24:49.573784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573762 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" Sep 07 09:24:49.573784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573767 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" Sep 07 09:24:49.573784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573775 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" Sep 07 09:24:49.573784 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573781 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" Sep 07 09:24:49.573996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573794 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" Sep 07 09:24:49.573996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573800 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c6e8ab3-c705-42c2-a76b-435a3023e217" containerName="istio-proxy" Sep 07 09:24:49.573996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573874 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" Sep 07 09:24:49.573996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573882 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d37e491-4642-461c-a9bb-54ea0af1ad9d" containerName="revoke-keys" Sep 07 09:24:49.573996 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.573888 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c0cafbc-3e23-4a44-89f9-fd1afa0f7c49" containerName="istio-proxy" Sep 07 09:24:49.578416 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.578400 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" Sep 07 09:24:49.581562 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.581504 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:24:49.582021 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.581999 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:24:49.584147 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.584123 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6"] Sep 07 09:24:49.683563 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.683532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75870475-147b-4e79-bca5-e793d8ef2d7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6\" (UID: \"75870475-147b-4e79-bca5-e793d8ef2d7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" Sep 07 09:24:49.784614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.784583 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75870475-147b-4e79-bca5-e793d8ef2d7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6\" (UID: \"75870475-147b-4e79-bca5-e793d8ef2d7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" Sep 07 09:24:49.785229 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.785211 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75870475-147b-4e79-bca5-e793d8ef2d7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6\" (UID: \"75870475-147b-4e79-bca5-e793d8ef2d7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" Sep 07 09:24:49.891428 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:49.891335 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" Sep 07 09:24:50.164623 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:50.164545 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6"] Sep 07 09:24:50.167104 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:24:50.167071 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75870475_147b_4e79_bca5_e793d8ef2d7a.slice/crio-8879d39f29939b31e27bd178fe20fca44693c9ecc56db76688bfee520d57f3d4 WatchSource:0}: Error finding container 8879d39f29939b31e27bd178fe20fca44693c9ecc56db76688bfee520d57f3d4: Status 404 returned error can't find the container with id 8879d39f29939b31e27bd178fe20fca44693c9ecc56db76688bfee520d57f3d4 Sep 07 09:24:50.716536 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:50.716451 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" event={"ID":"75870475-147b-4e79-bca5-e793d8ef2d7a","Type":"ContainerStarted","Data":"aaa7efe99388a1f140fec41f59d4651e62732f976e3fcd67d640814774b688e0"} Sep 07 09:24:50.716536 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:50.716536 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" event={"ID":"75870475-147b-4e79-bca5-e793d8ef2d7a","Type":"ContainerStarted","Data":"8879d39f29939b31e27bd178fe20fca44693c9ecc56db76688bfee520d57f3d4"} Sep 07 09:24:50.733635 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:50.733580 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" podStartSLOduration=1.733564627 podStartE2EDuration="1.733564627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:24:50.730915156 +0000 UTC m=+1549.247706060" watchObservedRunningTime="2026-09-07 09:24:50.733564627 +0000 UTC m=+1549.250355529" Sep 07 09:24:51.723728 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:51.723692 2558 generic.go:356] "Generic (PLEG): container finished" podID="75870475-147b-4e79-bca5-e793d8ef2d7a" containerID="aaa7efe99388a1f140fec41f59d4651e62732f976e3fcd67d640814774b688e0" exitCode=0 Sep 07 09:24:51.724108 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:51.723778 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" event={"ID":"75870475-147b-4e79-bca5-e793d8ef2d7a","Type":"ContainerDied","Data":"aaa7efe99388a1f140fec41f59d4651e62732f976e3fcd67d640814774b688e0"} Sep 07 09:24:52.878466 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:52.878442 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" Sep 07 09:24:53.014635 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:53.014558 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75870475-147b-4e79-bca5-e793d8ef2d7a-maas-api-service-ca\") pod \"75870475-147b-4e79-bca5-e793d8ef2d7a\" (UID: \"75870475-147b-4e79-bca5-e793d8ef2d7a\") " Sep 07 09:24:53.014897 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:53.014876 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75870475-147b-4e79-bca5-e793d8ef2d7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "75870475-147b-4e79-bca5-e793d8ef2d7a" (UID: "75870475-147b-4e79-bca5-e793d8ef2d7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:24:53.116211 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:53.116182 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75870475-147b-4e79-bca5-e793d8ef2d7a-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:24:53.736686 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:53.736656 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" Sep 07 09:24:53.736832 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:53.736688 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6" event={"ID":"75870475-147b-4e79-bca5-e793d8ef2d7a","Type":"ContainerDied","Data":"8879d39f29939b31e27bd178fe20fca44693c9ecc56db76688bfee520d57f3d4"} Sep 07 09:24:53.736832 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:24:53.736732 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8879d39f29939b31e27bd178fe20fca44693c9ecc56db76688bfee520d57f3d4" Sep 07 09:25:00.098195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:00.098152 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw"] Sep 07 09:25:00.232487 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:00.232456 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6"] Sep 07 09:25:00.234683 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:00.234660 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-5560ef-zxzz6"] Sep 07 09:25:00.777184 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:00.777146 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" podUID="d71b023f-f081-4224-a8fc-567f535c8ad8" containerName="payload-pre-processing" containerID="cri-o://b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e" gracePeriod=30 Sep 07 09:25:01.030520 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.030464 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:25:01.091553 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.091529 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm2cq\" (UniqueName: \"kubernetes.io/projected/d71b023f-f081-4224-a8fc-567f535c8ad8-kube-api-access-hm2cq\") pod \"d71b023f-f081-4224-a8fc-567f535c8ad8\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " Sep 07 09:25:01.091703 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.091642 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71b023f-f081-4224-a8fc-567f535c8ad8-plugins-config-volume\") pod \"d71b023f-f081-4224-a8fc-567f535c8ad8\" (UID: \"d71b023f-f081-4224-a8fc-567f535c8ad8\") " Sep 07 09:25:01.092004 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.091980 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d71b023f-f081-4224-a8fc-567f535c8ad8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d71b023f-f081-4224-a8fc-567f535c8ad8" (UID: "d71b023f-f081-4224-a8fc-567f535c8ad8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:25:01.093664 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.093636 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d71b023f-f081-4224-a8fc-567f535c8ad8-kube-api-access-hm2cq" (OuterVolumeSpecName: "kube-api-access-hm2cq") pod "d71b023f-f081-4224-a8fc-567f535c8ad8" (UID: "d71b023f-f081-4224-a8fc-567f535c8ad8"). InnerVolumeSpecName "kube-api-access-hm2cq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:25:01.193194 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.193163 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d71b023f-f081-4224-a8fc-567f535c8ad8-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:01.193194 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.193188 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm2cq\" (UniqueName: \"kubernetes.io/projected/d71b023f-f081-4224-a8fc-567f535c8ad8-kube-api-access-hm2cq\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:01.783587 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.783548 2558 generic.go:356] "Generic (PLEG): container finished" podID="d71b023f-f081-4224-a8fc-567f535c8ad8" containerID="b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e" exitCode=0 Sep 07 09:25:01.783765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.783591 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" Sep 07 09:25:01.783765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.783629 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" event={"ID":"d71b023f-f081-4224-a8fc-567f535c8ad8","Type":"ContainerDied","Data":"b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e"} Sep 07 09:25:01.783765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.783673 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw" event={"ID":"d71b023f-f081-4224-a8fc-567f535c8ad8","Type":"ContainerDied","Data":"5b2c5798d4693cdb082f3662adeb1f4aad809cb9b6b39e93889452a2d29c22b3"} Sep 07 09:25:01.783765 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.783697 2558 scope.go:117] "RemoveContainer" containerID="b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e" Sep 07 09:25:01.800208 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.800188 2558 scope.go:117] "RemoveContainer" containerID="b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e" Sep 07 09:25:01.800508 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:25:01.800487 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e\": container with ID starting with b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e not found: ID does not exist" containerID="b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e" Sep 07 09:25:01.800622 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.800514 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e"} err="failed to get container status \"b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e\": rpc error: code = NotFound desc = could not find container \"b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e\": container with ID starting with b110b00872103a52989f547e25738ac12d9528c6fc30c7cdc3b41b11e4938a2e not found: ID does not exist" Sep 07 09:25:01.808477 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.808453 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw"] Sep 07 09:25:01.812154 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.812132 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5560ef-67bb85546-vmdsw"] Sep 07 09:25:01.951348 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.951290 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75870475-147b-4e79-bca5-e793d8ef2d7a" path="/var/lib/kubelet/pods/75870475-147b-4e79-bca5-e793d8ef2d7a/volumes" Sep 07 09:25:01.951754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:01.951735 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d71b023f-f081-4224-a8fc-567f535c8ad8" path="/var/lib/kubelet/pods/d71b023f-f081-4224-a8fc-567f535c8ad8/volumes" Sep 07 09:25:02.243359 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:02.243276 2558 scope.go:117] "RemoveContainer" containerID="d9b31f5b858c3a9129adb13104c65f4ccf594780c11516f3e7015613838125c8" Sep 07 09:25:06.486531 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:06.486499 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz"] Sep 07 09:25:06.487656 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:06.487621 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" containerID="cri-o://ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277" gracePeriod=30 Sep 07 09:25:11.747755 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.747726 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:25:11.788040 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788008 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-data\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788058 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-credential-socket\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788112 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istiod-ca-cert\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788153 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-podinfo\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788201 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c99bj\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-kube-api-access-c99bj\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788226 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-token\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788255 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-data" (OuterVolumeSpecName: "istio-data") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:25:11.788400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788269 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-envoy\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788337 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:25:11.788400 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788374 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-certs\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788720 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788423 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-socket\") pod \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\" (UID: \"f3458db0-ee1d-40ec-8bc9-e695a820ce15\") " Sep 07 09:25:11.788720 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788590 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:25:11.788720 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788698 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:25:11.788885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788806 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:25:11.788957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788913 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-data\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.788957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788930 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-credential-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.788957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788943 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istiod-ca-cert\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.788957 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788957 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-certs\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.789167 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.788970 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-workload-socket\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.790416 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.790392 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 07 09:25:11.790625 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.790601 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-token" (OuterVolumeSpecName: "istio-token") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:25:11.790752 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.790731 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-kube-api-access-c99bj" (OuterVolumeSpecName: "kube-api-access-c99bj") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "kube-api-access-c99bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:25:11.790813 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.790798 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f3458db0-ee1d-40ec-8bc9-e695a820ce15" (UID: "f3458db0-ee1d-40ec-8bc9-e695a820ce15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 07 09:25:11.839896 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.839862 2558 generic.go:356] "Generic (PLEG): container finished" podID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerID="ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277" exitCode=0 Sep 07 09:25:11.840002 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.839916 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" Sep 07 09:25:11.840002 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.839949 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" event={"ID":"f3458db0-ee1d-40ec-8bc9-e695a820ce15","Type":"ContainerDied","Data":"ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277"} Sep 07 09:25:11.840090 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.840003 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz" event={"ID":"f3458db0-ee1d-40ec-8bc9-e695a820ce15","Type":"ContainerDied","Data":"39bc57903b102a1d406b9b4d00f3bb59f1c0a38631f52cbddad11c92eb5933a6"} Sep 07 09:25:11.840090 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.840026 2558 scope.go:117] "RemoveContainer" containerID="ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277" Sep 07 09:25:11.853392 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.853374 2558 scope.go:117] "RemoveContainer" containerID="7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7" Sep 07 09:25:11.861674 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.861651 2558 scope.go:117] "RemoveContainer" containerID="ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277" Sep 07 09:25:11.862150 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:25:11.862049 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277\": container with ID starting with ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277 not found: ID does not exist" containerID="ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277" Sep 07 09:25:11.862268 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.862157 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277"} err="failed to get container status \"ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277\": rpc error: code = NotFound desc = could not find container \"ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277\": container with ID starting with ba7424497173018ba4368acef4333e6f8ce2cd8b896ca5ab2e17046dde009277 not found: ID does not exist" Sep 07 09:25:11.862268 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.862190 2558 scope.go:117] "RemoveContainer" containerID="7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7" Sep 07 09:25:11.862839 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:25:11.862528 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7\": container with ID starting with 7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7 not found: ID does not exist" containerID="7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7" Sep 07 09:25:11.862839 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.862552 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7"} err="failed to get container status \"7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7\": rpc error: code = NotFound desc = could not find container \"7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7\": container with ID starting with 7bd66167ba5aa2e117c537f5a1b4d3c37ad18c0e6d7309ed7b16235348eb0de7 not found: ID does not exist" Sep 07 09:25:11.865260 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.865238 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz"] Sep 07 09:25:11.868392 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.868372 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5560ef-openshift-default-7765894878-brmqz"] Sep 07 09:25:11.889419 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.889399 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-podinfo\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.889419 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.889419 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c99bj\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-kube-api-access-c99bj\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.889545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.889429 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-token\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.889545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.889440 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3458db0-ee1d-40ec-8bc9-e695a820ce15-istio-envoy\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:25:11.951906 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:11.951871 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" path="/var/lib/kubelet/pods/f3458db0-ee1d-40ec-8bc9-e695a820ce15/volumes" Sep 07 09:25:33.308168 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308133 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn"] Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308517 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75870475-147b-4e79-bca5-e793d8ef2d7a" containerName="revoke-keys" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308529 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="75870475-147b-4e79-bca5-e793d8ef2d7a" containerName="revoke-keys" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308545 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308550 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308557 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d71b023f-f081-4224-a8fc-567f535c8ad8" containerName="payload-pre-processing" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308564 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d71b023f-f081-4224-a8fc-567f535c8ad8" containerName="payload-pre-processing" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308570 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308577 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308643 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d71b023f-f081-4224-a8fc-567f535c8ad8" containerName="payload-pre-processing" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308652 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308657 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="75870475-147b-4e79-bca5-e793d8ef2d7a" containerName="revoke-keys" Sep 07 09:25:33.308671 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.308665 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3458db0-ee1d-40ec-8bc9-e695a820ce15" containerName="istio-proxy" Sep 07 09:25:33.312813 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.312793 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.315292 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.315268 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-0a9a72\"" Sep 07 09:25:33.315512 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.315276 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-0a9a72-dockercfg-knvrq\"" Sep 07 09:25:33.320018 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.319990 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn"] Sep 07 09:25:33.329429 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.329408 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r"] Sep 07 09:25:33.332813 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.332794 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.343396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.343373 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r"] Sep 07 09:25:33.491214 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.491182 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbcb0f19-7643-4fbf-a103-b78b169ab65d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.491404 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.491230 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75z2\" (UniqueName: \"kubernetes.io/projected/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-kube-api-access-j75z2\") pod \"payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.491404 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.491351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.491483 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.491411 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvzg2\" (UniqueName: \"kubernetes.io/projected/dbcb0f19-7643-4fbf-a103-b78b169ab65d-kube-api-access-mvzg2\") pod \"payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.592498 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.592413 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75z2\" (UniqueName: \"kubernetes.io/projected/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-kube-api-access-j75z2\") pod \"payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.592498 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.592490 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.592682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.592519 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvzg2\" (UniqueName: \"kubernetes.io/projected/dbcb0f19-7643-4fbf-a103-b78b169ab65d-kube-api-access-mvzg2\") pod \"payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.592682 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.592608 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbcb0f19-7643-4fbf-a103-b78b169ab65d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.593273 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.593252 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.593339 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.593257 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbcb0f19-7643-4fbf-a103-b78b169ab65d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.599951 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.599923 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvzg2\" (UniqueName: \"kubernetes.io/projected/dbcb0f19-7643-4fbf-a103-b78b169ab65d-kube-api-access-mvzg2\") pod \"payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.600147 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.600129 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75z2\" (UniqueName: \"kubernetes.io/projected/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-kube-api-access-j75z2\") pod \"payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.625120 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.625095 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:33.644750 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.644663 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:33.902011 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.901922 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn"] Sep 07 09:25:33.909571 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:25:33.909546 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc5d0e6_19e9_45c8_8b08_41a4cb1544a2.slice/crio-37139e9318cb2262b0b174f607d44a4413a4e865fe086f1a0a411808929383ab WatchSource:0}: Error finding container 37139e9318cb2262b0b174f607d44a4413a4e865fe086f1a0a411808929383ab: Status 404 returned error can't find the container with id 37139e9318cb2262b0b174f607d44a4413a4e865fe086f1a0a411808929383ab Sep 07 09:25:33.932273 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.932246 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r"] Sep 07 09:25:33.942276 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:25:33.942248 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbcb0f19_7643_4fbf_a103_b78b169ab65d.slice/crio-31846103ddf141d18f15e0b9f862d099deff900f490ef33e42139429276f917d WatchSource:0}: Error finding container 31846103ddf141d18f15e0b9f862d099deff900f490ef33e42139429276f917d: Status 404 returned error can't find the container with id 31846103ddf141d18f15e0b9f862d099deff900f490ef33e42139429276f917d Sep 07 09:25:33.967927 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.967899 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" event={"ID":"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2","Type":"ContainerStarted","Data":"37139e9318cb2262b0b174f607d44a4413a4e865fe086f1a0a411808929383ab"} Sep 07 09:25:33.969076 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:33.969055 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" event={"ID":"dbcb0f19-7643-4fbf-a103-b78b169ab65d","Type":"ContainerStarted","Data":"31846103ddf141d18f15e0b9f862d099deff900f490ef33e42139429276f917d"} Sep 07 09:25:34.977478 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:34.977445 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" event={"ID":"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2","Type":"ContainerStarted","Data":"950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b"} Sep 07 09:25:34.977891 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:34.977505 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:34.979113 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:34.979090 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" event={"ID":"dbcb0f19-7643-4fbf-a103-b78b169ab65d","Type":"ContainerStarted","Data":"6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd"} Sep 07 09:25:34.979190 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:34.979133 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:25:34.996216 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:34.996176 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" podStartSLOduration=1.996162055 podStartE2EDuration="1.996162055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:25:34.994320012 +0000 UTC m=+1593.511110937" watchObservedRunningTime="2026-09-07 09:25:34.996162055 +0000 UTC m=+1593.512952959" Sep 07 09:25:35.010663 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:35.010615 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" podStartSLOduration=2.010601082 podStartE2EDuration="2.010601082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:25:35.008220276 +0000 UTC m=+1593.525011180" watchObservedRunningTime="2026-09-07 09:25:35.010601082 +0000 UTC m=+1593.527391984" Sep 07 09:25:45.986690 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:45.986659 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:25:45.987085 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:25:45.986829 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:26:02.393064 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.393034 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47"] Sep 07 09:26:02.398515 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.398488 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.401122 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.401099 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-da0a32-dockercfg-zhgq2\"" Sep 07 09:26:02.401259 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.401102 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-da0a32\"" Sep 07 09:26:02.413327 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.411797 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47"] Sep 07 09:26:02.453367 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.453327 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dssbv\" (UniqueName: \"kubernetes.io/projected/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-kube-api-access-dssbv\") pod \"payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.453521 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.453389 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.554197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.554089 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dssbv\" (UniqueName: \"kubernetes.io/projected/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-kube-api-access-dssbv\") pod \"payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.554197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.554147 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.554887 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.554864 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.562875 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.562841 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dssbv\" (UniqueName: \"kubernetes.io/projected/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-kube-api-access-dssbv\") pod \"payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.719518 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.719466 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:02.989557 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:02.989526 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47"] Sep 07 09:26:02.990997 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:26:02.990967 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a3c15a_d5aa_4da2_8a9d_d8db9407afdf.slice/crio-42dfbd7f9290139ee90745b1f033a324160180208835a6629fc97bb5a0c078b6 WatchSource:0}: Error finding container 42dfbd7f9290139ee90745b1f033a324160180208835a6629fc97bb5a0c078b6: Status 404 returned error can't find the container with id 42dfbd7f9290139ee90745b1f033a324160180208835a6629fc97bb5a0c078b6 Sep 07 09:26:03.150314 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:03.150270 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" event={"ID":"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf","Type":"ContainerStarted","Data":"aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64"} Sep 07 09:26:03.150492 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:03.150331 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" event={"ID":"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf","Type":"ContainerStarted","Data":"42dfbd7f9290139ee90745b1f033a324160180208835a6629fc97bb5a0c078b6"} Sep 07 09:26:03.150492 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:03.150431 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:03.168514 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:03.168460 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" podStartSLOduration=1.168443786 podStartE2EDuration="1.168443786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:26:03.165934448 +0000 UTC m=+1621.682725352" watchObservedRunningTime="2026-09-07 09:26:03.168443786 +0000 UTC m=+1621.685234688" Sep 07 09:26:14.158511 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:14.158481 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:26:30.771962 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:26:30.771933 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47_58a3c15a-d5aa-4da2-8a9d-d8db9407afdf/payload-processing/0.log" Sep 07 09:27:02.305420 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:02.305389 2558 scope.go:117] "RemoveContainer" containerID="edb7d54a6f83c48be4b050d0fd7c2911180d939547cf939d37e7db01f7e91036" Sep 07 09:27:08.437876 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:08.437843 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r_dbcb0f19-7643-4fbf-a103-b78b169ab65d/payload-processing/0.log" Sep 07 09:27:08.670627 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:08.670599 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47_58a3c15a-d5aa-4da2-8a9d-d8db9407afdf/payload-processing/0.log" Sep 07 09:27:13.725262 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.725221 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2"] Sep 07 09:27:13.728949 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.728927 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:13.731991 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.731966 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d9283c-dockercfg-f59hm\"" Sep 07 09:27:13.732129 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.732113 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d9283c\"" Sep 07 09:27:13.740128 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.740105 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2"] Sep 07 09:27:13.823009 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.822968 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxlqx\" (UniqueName: \"kubernetes.io/projected/5f947908-cb5b-4e86-b5a0-875985a80786-kube-api-access-mxlqx\") pod \"payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:13.823197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.823086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f947908-cb5b-4e86-b5a0-875985a80786-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:13.924454 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.924412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f947908-cb5b-4e86-b5a0-875985a80786-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:13.924632 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.924510 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxlqx\" (UniqueName: \"kubernetes.io/projected/5f947908-cb5b-4e86-b5a0-875985a80786-kube-api-access-mxlqx\") pod \"payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:13.925141 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.925120 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f947908-cb5b-4e86-b5a0-875985a80786-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:13.933574 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:13.933552 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxlqx\" (UniqueName: \"kubernetes.io/projected/5f947908-cb5b-4e86-b5a0-875985a80786-kube-api-access-mxlqx\") pod \"payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:14.040056 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:14.040024 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:14.310469 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:14.310442 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2"] Sep 07 09:27:14.311934 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:27:14.311910 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f947908_cb5b_4e86_b5a0_875985a80786.slice/crio-0e9bb9cf051a910dbbbacc1aec740edc1b632d6ab8672d21335367ebcfbba793 WatchSource:0}: Error finding container 0e9bb9cf051a910dbbbacc1aec740edc1b632d6ab8672d21335367ebcfbba793: Status 404 returned error can't find the container with id 0e9bb9cf051a910dbbbacc1aec740edc1b632d6ab8672d21335367ebcfbba793 Sep 07 09:27:14.570598 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:14.570504 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" event={"ID":"5f947908-cb5b-4e86-b5a0-875985a80786","Type":"ContainerStarted","Data":"2f0f233863552bf36c9352e341bbe952a18e61c76b968ac79e47f65c76f41eac"} Sep 07 09:27:14.570598 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:14.570568 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" event={"ID":"5f947908-cb5b-4e86-b5a0-875985a80786","Type":"ContainerStarted","Data":"0e9bb9cf051a910dbbbacc1aec740edc1b632d6ab8672d21335367ebcfbba793"} Sep 07 09:27:14.570598 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:14.570598 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:14.587408 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:14.587360 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" podStartSLOduration=1.587345367 podStartE2EDuration="1.587345367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:27:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:27:14.585447926 +0000 UTC m=+1693.102238840" watchObservedRunningTime="2026-09-07 09:27:14.587345367 +0000 UTC m=+1693.104136278" Sep 07 09:27:25.579118 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:25.579039 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:37.631791 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.631760 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms"] Sep 07 09:27:37.635389 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.635369 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" Sep 07 09:27:37.637885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.637862 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:27:37.637885 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.637870 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:27:37.644324 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.644283 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms"] Sep 07 09:27:37.746869 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.746835 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249345fb-ab1e-4c84-bebf-cabbbe062425-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms\" (UID: \"249345fb-ab1e-4c84-bebf-cabbbe062425\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" Sep 07 09:27:37.847497 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.847464 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249345fb-ab1e-4c84-bebf-cabbbe062425-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms\" (UID: \"249345fb-ab1e-4c84-bebf-cabbbe062425\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" Sep 07 09:27:37.848096 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.848077 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249345fb-ab1e-4c84-bebf-cabbbe062425-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms\" (UID: \"249345fb-ab1e-4c84-bebf-cabbbe062425\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" Sep 07 09:27:37.947583 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:37.947496 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" Sep 07 09:27:38.221348 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:38.221315 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms"] Sep 07 09:27:38.222368 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:27:38.222338 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249345fb_ab1e_4c84_bebf_cabbbe062425.slice/crio-18fdf1414973d25a3d5e9ee4e81157f303ddd6e9f9d99bd8f0814ab01f8f7534 WatchSource:0}: Error finding container 18fdf1414973d25a3d5e9ee4e81157f303ddd6e9f9d99bd8f0814ab01f8f7534: Status 404 returned error can't find the container with id 18fdf1414973d25a3d5e9ee4e81157f303ddd6e9f9d99bd8f0814ab01f8f7534 Sep 07 09:27:38.713858 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:38.713820 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" event={"ID":"249345fb-ab1e-4c84-bebf-cabbbe062425","Type":"ContainerStarted","Data":"8823aba76c9b797e9dd674a3e5241603d9f2039c1af05a2047f1d97abd5594ad"} Sep 07 09:27:38.714236 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:38.713903 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" event={"ID":"249345fb-ab1e-4c84-bebf-cabbbe062425","Type":"ContainerStarted","Data":"18fdf1414973d25a3d5e9ee4e81157f303ddd6e9f9d99bd8f0814ab01f8f7534"} Sep 07 09:27:39.720583 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:39.720545 2558 generic.go:356] "Generic (PLEG): container finished" podID="249345fb-ab1e-4c84-bebf-cabbbe062425" containerID="8823aba76c9b797e9dd674a3e5241603d9f2039c1af05a2047f1d97abd5594ad" exitCode=0 Sep 07 09:27:39.721037 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:39.720631 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" event={"ID":"249345fb-ab1e-4c84-bebf-cabbbe062425","Type":"ContainerDied","Data":"8823aba76c9b797e9dd674a3e5241603d9f2039c1af05a2047f1d97abd5594ad"} Sep 07 09:27:40.863900 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:40.863878 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" Sep 07 09:27:40.977111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:40.977010 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249345fb-ab1e-4c84-bebf-cabbbe062425-maas-api-service-ca\") pod \"249345fb-ab1e-4c84-bebf-cabbbe062425\" (UID: \"249345fb-ab1e-4c84-bebf-cabbbe062425\") " Sep 07 09:27:40.977381 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:40.977361 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/249345fb-ab1e-4c84-bebf-cabbbe062425-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "249345fb-ab1e-4c84-bebf-cabbbe062425" (UID: "249345fb-ab1e-4c84-bebf-cabbbe062425"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:27:41.078438 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:41.078407 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/249345fb-ab1e-4c84-bebf-cabbbe062425-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:27:41.733396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:41.733358 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" event={"ID":"249345fb-ab1e-4c84-bebf-cabbbe062425","Type":"ContainerDied","Data":"18fdf1414973d25a3d5e9ee4e81157f303ddd6e9f9d99bd8f0814ab01f8f7534"} Sep 07 09:27:41.733396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:41.733394 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms" Sep 07 09:27:41.733660 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:41.733408 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18fdf1414973d25a3d5e9ee4e81157f303ddd6e9f9d99bd8f0814ab01f8f7534" Sep 07 09:27:43.568111 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.568076 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2"] Sep 07 09:27:43.568610 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.568320 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" podUID="5f947908-cb5b-4e86-b5a0-875985a80786" containerName="payload-processing" containerID="cri-o://2f0f233863552bf36c9352e341bbe952a18e61c76b968ac79e47f65c76f41eac" gracePeriod=30 Sep 07 09:27:43.752941 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.752904 2558 generic.go:356] "Generic (PLEG): container finished" podID="5f947908-cb5b-4e86-b5a0-875985a80786" containerID="2f0f233863552bf36c9352e341bbe952a18e61c76b968ac79e47f65c76f41eac" exitCode=0 Sep 07 09:27:43.753127 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.753011 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" event={"ID":"5f947908-cb5b-4e86-b5a0-875985a80786","Type":"ContainerDied","Data":"2f0f233863552bf36c9352e341bbe952a18e61c76b968ac79e47f65c76f41eac"} Sep 07 09:27:43.773977 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.773950 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms"] Sep 07 09:27:43.776365 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.776342 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d9283c-4snms"] Sep 07 09:27:43.836403 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.836291 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:43.953198 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:43.953165 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="249345fb-ab1e-4c84-bebf-cabbbe062425" path="/var/lib/kubelet/pods/249345fb-ab1e-4c84-bebf-cabbbe062425/volumes" Sep 07 09:27:44.004555 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.004523 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxlqx\" (UniqueName: \"kubernetes.io/projected/5f947908-cb5b-4e86-b5a0-875985a80786-kube-api-access-mxlqx\") pod \"5f947908-cb5b-4e86-b5a0-875985a80786\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " Sep 07 09:27:44.004715 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.004704 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f947908-cb5b-4e86-b5a0-875985a80786-plugins-config-volume\") pod \"5f947908-cb5b-4e86-b5a0-875985a80786\" (UID: \"5f947908-cb5b-4e86-b5a0-875985a80786\") " Sep 07 09:27:44.005084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.005059 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f947908-cb5b-4e86-b5a0-875985a80786-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f947908-cb5b-4e86-b5a0-875985a80786" (UID: "5f947908-cb5b-4e86-b5a0-875985a80786"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:27:44.006680 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.006657 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f947908-cb5b-4e86-b5a0-875985a80786-kube-api-access-mxlqx" (OuterVolumeSpecName: "kube-api-access-mxlqx") pod "5f947908-cb5b-4e86-b5a0-875985a80786" (UID: "5f947908-cb5b-4e86-b5a0-875985a80786"). InnerVolumeSpecName "kube-api-access-mxlqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:27:44.106382 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.106283 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f947908-cb5b-4e86-b5a0-875985a80786-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:27:44.106382 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.106333 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxlqx\" (UniqueName: \"kubernetes.io/projected/5f947908-cb5b-4e86-b5a0-875985a80786-kube-api-access-mxlqx\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:27:44.760416 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.760386 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" Sep 07 09:27:44.760840 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.760431 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2" event={"ID":"5f947908-cb5b-4e86-b5a0-875985a80786","Type":"ContainerDied","Data":"0e9bb9cf051a910dbbbacc1aec740edc1b632d6ab8672d21335367ebcfbba793"} Sep 07 09:27:44.760840 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.760497 2558 scope.go:117] "RemoveContainer" containerID="2f0f233863552bf36c9352e341bbe952a18e61c76b968ac79e47f65c76f41eac" Sep 07 09:27:44.782278 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.782248 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2"] Sep 07 09:27:44.784657 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:44.784637 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d9283c-7df57bbff7-gh8r2"] Sep 07 09:27:45.952055 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:45.952023 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f947908-cb5b-4e86-b5a0-875985a80786" path="/var/lib/kubelet/pods/5f947908-cb5b-4e86-b5a0-875985a80786/volumes" Sep 07 09:27:53.356717 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.356679 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl"] Sep 07 09:27:53.357103 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.357090 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f947908-cb5b-4e86-b5a0-875985a80786" containerName="payload-processing" Sep 07 09:27:53.357149 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.357105 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f947908-cb5b-4e86-b5a0-875985a80786" containerName="payload-processing" Sep 07 09:27:53.357149 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.357121 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="249345fb-ab1e-4c84-bebf-cabbbe062425" containerName="revoke-keys" Sep 07 09:27:53.357149 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.357128 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="249345fb-ab1e-4c84-bebf-cabbbe062425" containerName="revoke-keys" Sep 07 09:27:53.357249 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.357207 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="249345fb-ab1e-4c84-bebf-cabbbe062425" containerName="revoke-keys" Sep 07 09:27:53.357249 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.357216 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f947908-cb5b-4e86-b5a0-875985a80786" containerName="payload-processing" Sep 07 09:27:53.361602 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.361585 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" Sep 07 09:27:53.364604 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.364574 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:27:53.365002 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.364612 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:27:53.367006 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.366979 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl"] Sep 07 09:27:53.387345 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.387315 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9344bcf-c342-4d7f-8412-0d87f41e4d17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl\" (UID: \"a9344bcf-c342-4d7f-8412-0d87f41e4d17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" Sep 07 09:27:53.487926 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.487893 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9344bcf-c342-4d7f-8412-0d87f41e4d17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl\" (UID: \"a9344bcf-c342-4d7f-8412-0d87f41e4d17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" Sep 07 09:27:53.488609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.488581 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9344bcf-c342-4d7f-8412-0d87f41e4d17-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl\" (UID: \"a9344bcf-c342-4d7f-8412-0d87f41e4d17\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" Sep 07 09:27:53.673550 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.673465 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" Sep 07 09:27:53.947146 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:27:53.947105 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9344bcf_c342_4d7f_8412_0d87f41e4d17.slice/crio-946f9d89421458ede47351ed14e7cc5b63c3f425416862bcd941cfb5165f0c00 WatchSource:0}: Error finding container 946f9d89421458ede47351ed14e7cc5b63c3f425416862bcd941cfb5165f0c00: Status 404 returned error can't find the container with id 946f9d89421458ede47351ed14e7cc5b63c3f425416862bcd941cfb5165f0c00 Sep 07 09:27:53.952622 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:53.952599 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl"] Sep 07 09:27:54.822737 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:54.822639 2558 generic.go:356] "Generic (PLEG): container finished" podID="a9344bcf-c342-4d7f-8412-0d87f41e4d17" containerID="8d4459800c26d1bdc8d6e1b6576dc10ab0b7950958c0e1c723682408de2b0cf0" exitCode=0 Sep 07 09:27:54.823140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:54.822726 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" event={"ID":"a9344bcf-c342-4d7f-8412-0d87f41e4d17","Type":"ContainerDied","Data":"8d4459800c26d1bdc8d6e1b6576dc10ab0b7950958c0e1c723682408de2b0cf0"} Sep 07 09:27:54.823140 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:54.822809 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" event={"ID":"a9344bcf-c342-4d7f-8412-0d87f41e4d17","Type":"ContainerStarted","Data":"946f9d89421458ede47351ed14e7cc5b63c3f425416862bcd941cfb5165f0c00"} Sep 07 09:27:55.962754 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:55.962731 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" Sep 07 09:27:56.009075 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:56.009050 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9344bcf-c342-4d7f-8412-0d87f41e4d17-maas-api-service-ca\") pod \"a9344bcf-c342-4d7f-8412-0d87f41e4d17\" (UID: \"a9344bcf-c342-4d7f-8412-0d87f41e4d17\") " Sep 07 09:27:56.009476 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:56.009439 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9344bcf-c342-4d7f-8412-0d87f41e4d17-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9344bcf-c342-4d7f-8412-0d87f41e4d17" (UID: "a9344bcf-c342-4d7f-8412-0d87f41e4d17"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:27:56.009609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:56.009591 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9344bcf-c342-4d7f-8412-0d87f41e4d17-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:27:56.835547 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:56.835518 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" Sep 07 09:27:56.835721 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:56.835561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl" event={"ID":"a9344bcf-c342-4d7f-8412-0d87f41e4d17","Type":"ContainerDied","Data":"946f9d89421458ede47351ed14e7cc5b63c3f425416862bcd941cfb5165f0c00"} Sep 07 09:27:56.835721 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:27:56.835610 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="946f9d89421458ede47351ed14e7cc5b63c3f425416862bcd941cfb5165f0c00" Sep 07 09:28:08.835614 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:08.835574 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r"] Sep 07 09:28:08.836110 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:08.835856 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" podUID="dbcb0f19-7643-4fbf-a103-b78b169ab65d" containerName="payload-processing" containerID="cri-o://6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd" gracePeriod=30 Sep 07 09:28:08.840890 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:08.840859 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn"] Sep 07 09:28:08.841160 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:08.841087 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" podUID="7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" containerName="payload-pre-processing" containerID="cri-o://950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b" gracePeriod=30 Sep 07 09:28:08.993179 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:08.993140 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl"] Sep 07 09:28:08.997379 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:08.997349 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-0a9a72-lzvcl"] Sep 07 09:28:09.134449 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.134425 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:28:09.137759 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.137730 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:28:09.225244 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.225208 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvzg2\" (UniqueName: \"kubernetes.io/projected/dbcb0f19-7643-4fbf-a103-b78b169ab65d-kube-api-access-mvzg2\") pod \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " Sep 07 09:28:09.225447 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.225284 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-plugins-config-volume\") pod \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " Sep 07 09:28:09.225447 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.225326 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbcb0f19-7643-4fbf-a103-b78b169ab65d-plugins-config-volume\") pod \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\" (UID: \"dbcb0f19-7643-4fbf-a103-b78b169ab65d\") " Sep 07 09:28:09.225447 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.225353 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j75z2\" (UniqueName: \"kubernetes.io/projected/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-kube-api-access-j75z2\") pod \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\" (UID: \"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2\") " Sep 07 09:28:09.225713 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.225683 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" (UID: "7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:28:09.225838 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.225736 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbcb0f19-7643-4fbf-a103-b78b169ab65d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbcb0f19-7643-4fbf-a103-b78b169ab65d" (UID: "dbcb0f19-7643-4fbf-a103-b78b169ab65d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:28:09.227428 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.227399 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-kube-api-access-j75z2" (OuterVolumeSpecName: "kube-api-access-j75z2") pod "7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" (UID: "7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2"). InnerVolumeSpecName "kube-api-access-j75z2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:28:09.227528 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.227466 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcb0f19-7643-4fbf-a103-b78b169ab65d-kube-api-access-mvzg2" (OuterVolumeSpecName: "kube-api-access-mvzg2") pod "dbcb0f19-7643-4fbf-a103-b78b169ab65d" (UID: "dbcb0f19-7643-4fbf-a103-b78b169ab65d"). InnerVolumeSpecName "kube-api-access-mvzg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:28:09.326042 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.326011 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvzg2\" (UniqueName: \"kubernetes.io/projected/dbcb0f19-7643-4fbf-a103-b78b169ab65d-kube-api-access-mvzg2\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:28:09.326042 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.326037 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:28:09.326042 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.326046 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbcb0f19-7643-4fbf-a103-b78b169ab65d-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:28:09.326270 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.326056 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j75z2\" (UniqueName: \"kubernetes.io/projected/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2-kube-api-access-j75z2\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:28:09.912730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.912691 2558 generic.go:356] "Generic (PLEG): container finished" podID="dbcb0f19-7643-4fbf-a103-b78b169ab65d" containerID="6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd" exitCode=0 Sep 07 09:28:09.913143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.912737 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" Sep 07 09:28:09.913143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.912776 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" event={"ID":"dbcb0f19-7643-4fbf-a103-b78b169ab65d","Type":"ContainerDied","Data":"6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd"} Sep 07 09:28:09.913143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.912833 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r" event={"ID":"dbcb0f19-7643-4fbf-a103-b78b169ab65d","Type":"ContainerDied","Data":"31846103ddf141d18f15e0b9f862d099deff900f490ef33e42139429276f917d"} Sep 07 09:28:09.913143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.912856 2558 scope.go:117] "RemoveContainer" containerID="6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd" Sep 07 09:28:09.914455 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.914436 2558 generic.go:356] "Generic (PLEG): container finished" podID="7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" containerID="950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b" exitCode=0 Sep 07 09:28:09.914554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.914476 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" Sep 07 09:28:09.914554 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.914518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" event={"ID":"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2","Type":"ContainerDied","Data":"950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b"} Sep 07 09:28:09.914643 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.914561 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn" event={"ID":"7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2","Type":"ContainerDied","Data":"37139e9318cb2262b0b174f607d44a4413a4e865fe086f1a0a411808929383ab"} Sep 07 09:28:09.923084 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.923067 2558 scope.go:117] "RemoveContainer" containerID="6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd" Sep 07 09:28:09.923397 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:28:09.923373 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd\": container with ID starting with 6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd not found: ID does not exist" containerID="6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd" Sep 07 09:28:09.923493 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.923408 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd"} err="failed to get container status \"6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd\": rpc error: code = NotFound desc = could not find container \"6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd\": container with ID starting with 6d74085fd45081a46f094abf27ec088b3c6152aadb20ea1c58d0ccf3fe9da8fd not found: ID does not exist" Sep 07 09:28:09.923493 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.923431 2558 scope.go:117] "RemoveContainer" containerID="950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b" Sep 07 09:28:09.931443 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.931367 2558 scope.go:117] "RemoveContainer" containerID="950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b" Sep 07 09:28:09.931661 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:28:09.931636 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b\": container with ID starting with 950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b not found: ID does not exist" containerID="950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b" Sep 07 09:28:09.931783 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.931674 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b"} err="failed to get container status \"950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b\": rpc error: code = NotFound desc = could not find container \"950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b\": container with ID starting with 950903cccd81177c8c877409a7159a452263efc37c2d49c42a61ae7b6241a96b not found: ID does not exist" Sep 07 09:28:09.934748 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.934725 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r"] Sep 07 09:28:09.939090 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.939070 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-0a9a72-85bf4dd6b9-vk44r"] Sep 07 09:28:09.952408 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.952385 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9344bcf-c342-4d7f-8412-0d87f41e4d17" path="/var/lib/kubelet/pods/a9344bcf-c342-4d7f-8412-0d87f41e4d17/volumes" Sep 07 09:28:09.952764 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.952750 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbcb0f19-7643-4fbf-a103-b78b169ab65d" path="/var/lib/kubelet/pods/dbcb0f19-7643-4fbf-a103-b78b169ab65d/volumes" Sep 07 09:28:09.953015 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.953003 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn"] Sep 07 09:28:09.953292 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:09.953276 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-0a9a72-5c9677c56b-jn5gn"] Sep 07 09:28:11.951251 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:11.951217 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" path="/var/lib/kubelet/pods/7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2/volumes" Sep 07 09:28:57.173892 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.173810 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq"] Sep 07 09:28:57.174317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174182 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" containerName="payload-pre-processing" Sep 07 09:28:57.174317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174194 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" containerName="payload-pre-processing" Sep 07 09:28:57.174317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174229 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcb0f19-7643-4fbf-a103-b78b169ab65d" containerName="payload-processing" Sep 07 09:28:57.174317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174234 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcb0f19-7643-4fbf-a103-b78b169ab65d" containerName="payload-processing" Sep 07 09:28:57.174317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174241 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9344bcf-c342-4d7f-8412-0d87f41e4d17" containerName="revoke-keys" Sep 07 09:28:57.174317 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174248 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9344bcf-c342-4d7f-8412-0d87f41e4d17" containerName="revoke-keys" Sep 07 09:28:57.174545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174322 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9344bcf-c342-4d7f-8412-0d87f41e4d17" containerName="revoke-keys" Sep 07 09:28:57.174545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174331 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcb0f19-7643-4fbf-a103-b78b169ab65d" containerName="payload-processing" Sep 07 09:28:57.174545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.174340 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bc5d0e6-19e9-45c8-8b08-41a4cb1544a2" containerName="payload-pre-processing" Sep 07 09:28:57.177430 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.177415 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" Sep 07 09:28:57.180042 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.180020 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:28:57.180181 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.180059 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:28:57.185261 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.185036 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq"] Sep 07 09:28:57.258904 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.258876 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd5320-8157-4a05-9085-352ec4308441-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq\" (UID: \"c7dd5320-8157-4a05-9085-352ec4308441\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" Sep 07 09:28:57.360331 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.360246 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd5320-8157-4a05-9085-352ec4308441-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq\" (UID: \"c7dd5320-8157-4a05-9085-352ec4308441\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" Sep 07 09:28:57.360909 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.360886 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd5320-8157-4a05-9085-352ec4308441-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq\" (UID: \"c7dd5320-8157-4a05-9085-352ec4308441\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" Sep 07 09:28:57.489099 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.489016 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" Sep 07 09:28:57.763279 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:57.763255 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq"] Sep 07 09:28:57.764980 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:28:57.764947 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7dd5320_8157_4a05_9085_352ec4308441.slice/crio-4fb0220c7573e35175899e5d945d55a569f50a16b9b660c7dbea89522744314a WatchSource:0}: Error finding container 4fb0220c7573e35175899e5d945d55a569f50a16b9b660c7dbea89522744314a: Status 404 returned error can't find the container with id 4fb0220c7573e35175899e5d945d55a569f50a16b9b660c7dbea89522744314a Sep 07 09:28:58.218737 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:58.218690 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" event={"ID":"c7dd5320-8157-4a05-9085-352ec4308441","Type":"ContainerStarted","Data":"8920dc8ac0ec708cec12db299ea3ece431a6f3ba51f0538de8892aff9d659e78"} Sep 07 09:28:58.219177 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:58.218758 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" event={"ID":"c7dd5320-8157-4a05-9085-352ec4308441","Type":"ContainerStarted","Data":"4fb0220c7573e35175899e5d945d55a569f50a16b9b660c7dbea89522744314a"} Sep 07 09:28:58.235536 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:58.235488 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" podStartSLOduration=1.235472146 podStartE2EDuration="1.235472146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:28:58.233457872 +0000 UTC m=+1796.750248778" watchObservedRunningTime="2026-09-07 09:28:58.235472146 +0000 UTC m=+1796.752263048" Sep 07 09:28:59.231781 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:59.231738 2558 generic.go:356] "Generic (PLEG): container finished" podID="c7dd5320-8157-4a05-9085-352ec4308441" containerID="8920dc8ac0ec708cec12db299ea3ece431a6f3ba51f0538de8892aff9d659e78" exitCode=0 Sep 07 09:28:59.232550 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:28:59.232518 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" event={"ID":"c7dd5320-8157-4a05-9085-352ec4308441","Type":"ContainerDied","Data":"8920dc8ac0ec708cec12db299ea3ece431a6f3ba51f0538de8892aff9d659e78"} Sep 07 09:29:00.385342 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:00.385318 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" Sep 07 09:29:00.488081 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:00.488048 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd5320-8157-4a05-9085-352ec4308441-maas-api-service-ca\") pod \"c7dd5320-8157-4a05-9085-352ec4308441\" (UID: \"c7dd5320-8157-4a05-9085-352ec4308441\") " Sep 07 09:29:00.488454 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:00.488404 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7dd5320-8157-4a05-9085-352ec4308441-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7dd5320-8157-4a05-9085-352ec4308441" (UID: "c7dd5320-8157-4a05-9085-352ec4308441"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:29:00.589645 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:00.589610 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7dd5320-8157-4a05-9085-352ec4308441-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:01.245086 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:01.245058 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" Sep 07 09:29:01.245252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:01.245080 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq" event={"ID":"c7dd5320-8157-4a05-9085-352ec4308441","Type":"ContainerDied","Data":"4fb0220c7573e35175899e5d945d55a569f50a16b9b660c7dbea89522744314a"} Sep 07 09:29:01.245252 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:01.245133 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fb0220c7573e35175899e5d945d55a569f50a16b9b660c7dbea89522744314a" Sep 07 09:29:02.081523 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:02.081498 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:29:02.083736 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:02.083709 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:29:07.694455 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:07.694420 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47"] Sep 07 09:29:07.694914 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:07.694638 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" podUID="58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" containerName="payload-processing" containerID="cri-o://aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64" gracePeriod=30 Sep 07 09:29:07.854273 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:07.854238 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq"] Sep 07 09:29:07.857480 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:07.857457 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-da0a32-sg6hq"] Sep 07 09:29:07.952180 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:07.952146 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7dd5320-8157-4a05-9085-352ec4308441" path="/var/lib/kubelet/pods/c7dd5320-8157-4a05-9085-352ec4308441/volumes" Sep 07 09:29:07.963206 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:07.963184 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:29:08.055995 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.055964 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dssbv\" (UniqueName: \"kubernetes.io/projected/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-kube-api-access-dssbv\") pod \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " Sep 07 09:29:08.056197 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.056081 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-plugins-config-volume\") pod \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\" (UID: \"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf\") " Sep 07 09:29:08.056479 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.056454 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" (UID: "58a3c15a-d5aa-4da2-8a9d-d8db9407afdf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:29:08.058225 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.058201 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-kube-api-access-dssbv" (OuterVolumeSpecName: "kube-api-access-dssbv") pod "58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" (UID: "58a3c15a-d5aa-4da2-8a9d-d8db9407afdf"). InnerVolumeSpecName "kube-api-access-dssbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:29:08.157817 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.157789 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dssbv\" (UniqueName: \"kubernetes.io/projected/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-kube-api-access-dssbv\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:08.157817 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.157815 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:08.285198 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.285164 2558 generic.go:356] "Generic (PLEG): container finished" podID="58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" containerID="aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64" exitCode=0 Sep 07 09:29:08.285396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.285210 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" Sep 07 09:29:08.285396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.285235 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" event={"ID":"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf","Type":"ContainerDied","Data":"aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64"} Sep 07 09:29:08.285396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.285284 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47" event={"ID":"58a3c15a-d5aa-4da2-8a9d-d8db9407afdf","Type":"ContainerDied","Data":"42dfbd7f9290139ee90745b1f033a324160180208835a6629fc97bb5a0c078b6"} Sep 07 09:29:08.285396 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.285343 2558 scope.go:117] "RemoveContainer" containerID="aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64" Sep 07 09:29:08.294459 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.294443 2558 scope.go:117] "RemoveContainer" containerID="aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64" Sep 07 09:29:08.294739 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:29:08.294713 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64\": container with ID starting with aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64 not found: ID does not exist" containerID="aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64" Sep 07 09:29:08.294843 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.294743 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64"} err="failed to get container status \"aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64\": rpc error: code = NotFound desc = could not find container \"aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64\": container with ID starting with aee4661f690f41c4f4efdf8a46713781ca41bc16e9dbc66758d9a4e275cd3c64 not found: ID does not exist" Sep 07 09:29:08.309468 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.309444 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47"] Sep 07 09:29:08.321953 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:08.321924 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-da0a32-69fb97f44f-lkj47"] Sep 07 09:29:09.951822 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:09.951787 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" path="/var/lib/kubelet/pods/58a3c15a-d5aa-4da2-8a9d-d8db9407afdf/volumes" Sep 07 09:29:14.844852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.841740 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d"] Sep 07 09:29:14.844852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.842590 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7dd5320-8157-4a05-9085-352ec4308441" containerName="revoke-keys" Sep 07 09:29:14.844852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.842611 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7dd5320-8157-4a05-9085-352ec4308441" containerName="revoke-keys" Sep 07 09:29:14.844852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.842652 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" containerName="payload-processing" Sep 07 09:29:14.844852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.842662 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" containerName="payload-processing" Sep 07 09:29:14.844852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.842843 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7dd5320-8157-4a05-9085-352ec4308441" containerName="revoke-keys" Sep 07 09:29:14.844852 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.842866 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a3c15a-d5aa-4da2-8a9d-d8db9407afdf" containerName="payload-processing" Sep 07 09:29:14.848753 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.848727 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" Sep 07 09:29:14.850541 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.850517 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d"] Sep 07 09:29:14.851092 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.851068 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:29:14.851201 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.851106 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:29:14.923951 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:14.923920 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fded9f7-b4b3-4a91-8e98-091177f6468f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d\" (UID: \"1fded9f7-b4b3-4a91-8e98-091177f6468f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" Sep 07 09:29:15.024876 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:15.024836 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fded9f7-b4b3-4a91-8e98-091177f6468f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d\" (UID: \"1fded9f7-b4b3-4a91-8e98-091177f6468f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" Sep 07 09:29:15.025505 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:15.025489 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fded9f7-b4b3-4a91-8e98-091177f6468f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d\" (UID: \"1fded9f7-b4b3-4a91-8e98-091177f6468f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" Sep 07 09:29:15.159684 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:15.159595 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" Sep 07 09:29:15.441104 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:15.441080 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d"] Sep 07 09:29:15.441986 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:29:15.441962 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fded9f7_b4b3_4a91_8e98_091177f6468f.slice/crio-114a2d3022c39d868070ba655f27d613e0629343e201bd2d6c347ee30b8b8806 WatchSource:0}: Error finding container 114a2d3022c39d868070ba655f27d613e0629343e201bd2d6c347ee30b8b8806: Status 404 returned error can't find the container with id 114a2d3022c39d868070ba655f27d613e0629343e201bd2d6c347ee30b8b8806 Sep 07 09:29:16.337642 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:16.337609 2558 generic.go:356] "Generic (PLEG): container finished" podID="1fded9f7-b4b3-4a91-8e98-091177f6468f" containerID="65a24860a3e2ca4a3bee7aa971afd8cd9846114f6faac3571328c1d09c8ea909" exitCode=0 Sep 07 09:29:16.338001 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:16.337695 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" event={"ID":"1fded9f7-b4b3-4a91-8e98-091177f6468f","Type":"ContainerDied","Data":"65a24860a3e2ca4a3bee7aa971afd8cd9846114f6faac3571328c1d09c8ea909"} Sep 07 09:29:16.338001 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:16.337751 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" event={"ID":"1fded9f7-b4b3-4a91-8e98-091177f6468f","Type":"ContainerStarted","Data":"114a2d3022c39d868070ba655f27d613e0629343e201bd2d6c347ee30b8b8806"} Sep 07 09:29:17.475785 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:17.475760 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" Sep 07 09:29:17.648484 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:17.648392 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fded9f7-b4b3-4a91-8e98-091177f6468f-maas-api-service-ca\") pod \"1fded9f7-b4b3-4a91-8e98-091177f6468f\" (UID: \"1fded9f7-b4b3-4a91-8e98-091177f6468f\") " Sep 07 09:29:17.648757 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:17.648735 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fded9f7-b4b3-4a91-8e98-091177f6468f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1fded9f7-b4b3-4a91-8e98-091177f6468f" (UID: "1fded9f7-b4b3-4a91-8e98-091177f6468f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:29:17.749731 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:17.749699 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fded9f7-b4b3-4a91-8e98-091177f6468f-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:18.350249 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:18.350216 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" event={"ID":"1fded9f7-b4b3-4a91-8e98-091177f6468f","Type":"ContainerDied","Data":"114a2d3022c39d868070ba655f27d613e0629343e201bd2d6c347ee30b8b8806"} Sep 07 09:29:18.350416 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:18.350265 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="114a2d3022c39d868070ba655f27d613e0629343e201bd2d6c347ee30b8b8806" Sep 07 09:29:18.350416 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:18.350231 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d" Sep 07 09:29:25.261603 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.261568 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq"] Sep 07 09:29:25.262076 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.261807 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" podUID="e7435668-73f9-4c80-b9aa-3581dc97cebb" containerName="payload-processing" containerID="cri-o://9f9ce66c4bb750715641d040190f69bf702c0ce18ad6f054054cb4dfc84fbb64" gracePeriod=30 Sep 07 09:29:25.267414 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.267389 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5"] Sep 07 09:29:25.267651 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.267621 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" podUID="a15020df-8c55-4b68-8845-6cfb3dac4b15" containerName="payload-pre-processing" containerID="cri-o://0d62fefae190085e928574a733c94970e0a020cb14a1fe0a97c6944bb45ad9dc" gracePeriod=30 Sep 07 09:29:25.395677 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.395645 2558 generic.go:356] "Generic (PLEG): container finished" podID="e7435668-73f9-4c80-b9aa-3581dc97cebb" containerID="9f9ce66c4bb750715641d040190f69bf702c0ce18ad6f054054cb4dfc84fbb64" exitCode=0 Sep 07 09:29:25.395825 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.395713 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" event={"ID":"e7435668-73f9-4c80-b9aa-3581dc97cebb","Type":"ContainerDied","Data":"9f9ce66c4bb750715641d040190f69bf702c0ce18ad6f054054cb4dfc84fbb64"} Sep 07 09:29:25.397724 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.397703 2558 generic.go:356] "Generic (PLEG): container finished" podID="a15020df-8c55-4b68-8845-6cfb3dac4b15" containerID="0d62fefae190085e928574a733c94970e0a020cb14a1fe0a97c6944bb45ad9dc" exitCode=0 Sep 07 09:29:25.397826 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.397750 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" event={"ID":"a15020df-8c55-4b68-8845-6cfb3dac4b15","Type":"ContainerDied","Data":"0d62fefae190085e928574a733c94970e0a020cb14a1fe0a97c6944bb45ad9dc"} Sep 07 09:29:25.468205 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.468147 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d"] Sep 07 09:29:25.471450 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.471424 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3dc031-nf54d"] Sep 07 09:29:25.545270 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.542692 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:29:25.547926 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.547906 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:29:25.726597 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.726555 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whhgt\" (UniqueName: \"kubernetes.io/projected/a15020df-8c55-4b68-8845-6cfb3dac4b15-kube-api-access-whhgt\") pod \"a15020df-8c55-4b68-8845-6cfb3dac4b15\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " Sep 07 09:29:25.726787 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.726616 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7435668-73f9-4c80-b9aa-3581dc97cebb-plugins-config-volume\") pod \"e7435668-73f9-4c80-b9aa-3581dc97cebb\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " Sep 07 09:29:25.726787 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.726695 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76w24\" (UniqueName: \"kubernetes.io/projected/e7435668-73f9-4c80-b9aa-3581dc97cebb-kube-api-access-76w24\") pod \"e7435668-73f9-4c80-b9aa-3581dc97cebb\" (UID: \"e7435668-73f9-4c80-b9aa-3581dc97cebb\") " Sep 07 09:29:25.726883 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.726787 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a15020df-8c55-4b68-8845-6cfb3dac4b15-plugins-config-volume\") pod \"a15020df-8c55-4b68-8845-6cfb3dac4b15\" (UID: \"a15020df-8c55-4b68-8845-6cfb3dac4b15\") " Sep 07 09:29:25.727083 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.727051 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7435668-73f9-4c80-b9aa-3581dc97cebb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e7435668-73f9-4c80-b9aa-3581dc97cebb" (UID: "e7435668-73f9-4c80-b9aa-3581dc97cebb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:29:25.727206 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.727183 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15020df-8c55-4b68-8845-6cfb3dac4b15-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a15020df-8c55-4b68-8845-6cfb3dac4b15" (UID: "a15020df-8c55-4b68-8845-6cfb3dac4b15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:29:25.728726 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.728705 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7435668-73f9-4c80-b9aa-3581dc97cebb-kube-api-access-76w24" (OuterVolumeSpecName: "kube-api-access-76w24") pod "e7435668-73f9-4c80-b9aa-3581dc97cebb" (UID: "e7435668-73f9-4c80-b9aa-3581dc97cebb"). InnerVolumeSpecName "kube-api-access-76w24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:29:25.728893 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.728878 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a15020df-8c55-4b68-8845-6cfb3dac4b15-kube-api-access-whhgt" (OuterVolumeSpecName: "kube-api-access-whhgt") pod "a15020df-8c55-4b68-8845-6cfb3dac4b15" (UID: "a15020df-8c55-4b68-8845-6cfb3dac4b15"). InnerVolumeSpecName "kube-api-access-whhgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:29:25.827790 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.827704 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a15020df-8c55-4b68-8845-6cfb3dac4b15-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:25.827790 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.827733 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whhgt\" (UniqueName: \"kubernetes.io/projected/a15020df-8c55-4b68-8845-6cfb3dac4b15-kube-api-access-whhgt\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:25.827790 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.827743 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7435668-73f9-4c80-b9aa-3581dc97cebb-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:25.827790 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.827752 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76w24\" (UniqueName: \"kubernetes.io/projected/e7435668-73f9-4c80-b9aa-3581dc97cebb-kube-api-access-76w24\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:25.959166 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:25.959128 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fded9f7-b4b3-4a91-8e98-091177f6468f" path="/var/lib/kubelet/pods/1fded9f7-b4b3-4a91-8e98-091177f6468f/volumes" Sep 07 09:29:26.404225 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.404195 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" Sep 07 09:29:26.404697 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.404223 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq" event={"ID":"e7435668-73f9-4c80-b9aa-3581dc97cebb","Type":"ContainerDied","Data":"9995a719070521cadac6f9ee364cdd81669f70e56d6353f9a077951f5126d773"} Sep 07 09:29:26.404697 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.404291 2558 scope.go:117] "RemoveContainer" containerID="9f9ce66c4bb750715641d040190f69bf702c0ce18ad6f054054cb4dfc84fbb64" Sep 07 09:29:26.405802 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.405774 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" Sep 07 09:29:26.405879 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.405826 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5" event={"ID":"a15020df-8c55-4b68-8845-6cfb3dac4b15","Type":"ContainerDied","Data":"35f7a2d6890c2b1252507d72cbc959b2dc60f89769c5a46c605f122c06709dbb"} Sep 07 09:29:26.413558 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.413540 2558 scope.go:117] "RemoveContainer" containerID="0d62fefae190085e928574a733c94970e0a020cb14a1fe0a97c6944bb45ad9dc" Sep 07 09:29:26.424221 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.424202 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5"] Sep 07 09:29:26.429770 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.429749 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3dc031-845b688f5d-bdvc5"] Sep 07 09:29:26.443982 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.443957 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq"] Sep 07 09:29:26.446028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:26.446010 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-3dc031-7d96fb7495-lnhnq"] Sep 07 09:29:27.951654 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:27.951620 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a15020df-8c55-4b68-8845-6cfb3dac4b15" path="/var/lib/kubelet/pods/a15020df-8c55-4b68-8845-6cfb3dac4b15/volumes" Sep 07 09:29:27.952037 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:27.951979 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7435668-73f9-4c80-b9aa-3581dc97cebb" path="/var/lib/kubelet/pods/e7435668-73f9-4c80-b9aa-3581dc97cebb/volumes" Sep 07 09:29:32.143946 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.143913 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz"] Sep 07 09:29:32.144353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144269 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fded9f7-b4b3-4a91-8e98-091177f6468f" containerName="revoke-keys" Sep 07 09:29:32.144353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144280 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fded9f7-b4b3-4a91-8e98-091177f6468f" containerName="revoke-keys" Sep 07 09:29:32.144353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144309 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15020df-8c55-4b68-8845-6cfb3dac4b15" containerName="payload-pre-processing" Sep 07 09:29:32.144353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144315 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15020df-8c55-4b68-8845-6cfb3dac4b15" containerName="payload-pre-processing" Sep 07 09:29:32.144353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144324 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7435668-73f9-4c80-b9aa-3581dc97cebb" containerName="payload-processing" Sep 07 09:29:32.144353 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144330 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7435668-73f9-4c80-b9aa-3581dc97cebb" containerName="payload-processing" Sep 07 09:29:32.144555 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144386 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7435668-73f9-4c80-b9aa-3581dc97cebb" containerName="payload-processing" Sep 07 09:29:32.144555 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144396 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15020df-8c55-4b68-8845-6cfb3dac4b15" containerName="payload-pre-processing" Sep 07 09:29:32.144555 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.144405 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fded9f7-b4b3-4a91-8e98-091177f6468f" containerName="revoke-keys" Sep 07 09:29:32.147426 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.147411 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" Sep 07 09:29:32.149704 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.149678 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 07 09:29:32.149815 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.149704 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xz7zp\"" Sep 07 09:29:32.154217 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.154193 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz"] Sep 07 09:29:32.177429 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.177396 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f72ba771-294b-4785-a168-b5e9f3b5de03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz\" (UID: \"f72ba771-294b-4785-a168-b5e9f3b5de03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" Sep 07 09:29:32.277834 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.277799 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f72ba771-294b-4785-a168-b5e9f3b5de03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz\" (UID: \"f72ba771-294b-4785-a168-b5e9f3b5de03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" Sep 07 09:29:32.278428 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.278407 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f72ba771-294b-4785-a168-b5e9f3b5de03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz\" (UID: \"f72ba771-294b-4785-a168-b5e9f3b5de03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" Sep 07 09:29:32.458276 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.458192 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" Sep 07 09:29:32.731998 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:32.731969 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz"] Sep 07 09:29:32.733936 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:29:32.733913 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72ba771_294b_4785_a168_b5e9f3b5de03.slice/crio-57d75987c425576b2067cb1f21e66056afc13a102d780d1572179a5b238f7439 WatchSource:0}: Error finding container 57d75987c425576b2067cb1f21e66056afc13a102d780d1572179a5b238f7439: Status 404 returned error can't find the container with id 57d75987c425576b2067cb1f21e66056afc13a102d780d1572179a5b238f7439 Sep 07 09:29:33.453644 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:33.453609 2558 generic.go:356] "Generic (PLEG): container finished" podID="f72ba771-294b-4785-a168-b5e9f3b5de03" containerID="86e308004ae6299c78b4a74c9fc0b5c594b03f5ff57416617df0afb2a5d8c6a3" exitCode=0 Sep 07 09:29:33.453986 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:33.453669 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" event={"ID":"f72ba771-294b-4785-a168-b5e9f3b5de03","Type":"ContainerDied","Data":"86e308004ae6299c78b4a74c9fc0b5c594b03f5ff57416617df0afb2a5d8c6a3"} Sep 07 09:29:33.453986 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:33.453706 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" event={"ID":"f72ba771-294b-4785-a168-b5e9f3b5de03","Type":"ContainerStarted","Data":"57d75987c425576b2067cb1f21e66056afc13a102d780d1572179a5b238f7439"} Sep 07 09:29:34.593153 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:34.593128 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" Sep 07 09:29:34.593578 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:34.593563 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f72ba771-294b-4785-a168-b5e9f3b5de03-maas-api-service-ca\") pod \"f72ba771-294b-4785-a168-b5e9f3b5de03\" (UID: \"f72ba771-294b-4785-a168-b5e9f3b5de03\") " Sep 07 09:29:34.593882 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:34.593860 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f72ba771-294b-4785-a168-b5e9f3b5de03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f72ba771-294b-4785-a168-b5e9f3b5de03" (UID: "f72ba771-294b-4785-a168-b5e9f3b5de03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:29:34.694404 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:34.694376 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f72ba771-294b-4785-a168-b5e9f3b5de03-maas-api-service-ca\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:35.467067 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:35.467037 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" Sep 07 09:29:35.467244 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:35.467074 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz" event={"ID":"f72ba771-294b-4785-a168-b5e9f3b5de03","Type":"ContainerDied","Data":"57d75987c425576b2067cb1f21e66056afc13a102d780d1572179a5b238f7439"} Sep 07 09:29:35.467244 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:35.467124 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57d75987c425576b2067cb1f21e66056afc13a102d780d1572179a5b238f7439" Sep 07 09:29:42.530964 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.530932 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78"] Sep 07 09:29:42.531516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.531162 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" podUID="a7da42f5-56c3-4cc6-826c-c558bec8ff5f" containerName="payload-pre-processing" containerID="cri-o://846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122" gracePeriod=30 Sep 07 09:29:42.676605 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.676565 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz"] Sep 07 09:29:42.678949 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.678922 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34de95-nzdsz"] Sep 07 09:29:42.782507 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.782410 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:29:42.861391 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.861362 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmv2x\" (UniqueName: \"kubernetes.io/projected/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-kube-api-access-cmv2x\") pod \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " Sep 07 09:29:42.861567 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.861439 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-plugins-config-volume\") pod \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\" (UID: \"a7da42f5-56c3-4cc6-826c-c558bec8ff5f\") " Sep 07 09:29:42.861812 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.861790 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7da42f5-56c3-4cc6-826c-c558bec8ff5f" (UID: "a7da42f5-56c3-4cc6-826c-c558bec8ff5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 07 09:29:42.863421 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.863392 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-kube-api-access-cmv2x" (OuterVolumeSpecName: "kube-api-access-cmv2x") pod "a7da42f5-56c3-4cc6-826c-c558bec8ff5f" (UID: "a7da42f5-56c3-4cc6-826c-c558bec8ff5f"). InnerVolumeSpecName "kube-api-access-cmv2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 07 09:29:42.962780 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.962747 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmv2x\" (UniqueName: \"kubernetes.io/projected/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-kube-api-access-cmv2x\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:42.962780 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:42.962771 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7da42f5-56c3-4cc6-826c-c558bec8ff5f-plugins-config-volume\") on node \"ip-10-0-131-66.ec2.internal\" DevicePath \"\"" Sep 07 09:29:43.515088 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.515050 2558 generic.go:356] "Generic (PLEG): container finished" podID="a7da42f5-56c3-4cc6-826c-c558bec8ff5f" containerID="846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122" exitCode=0 Sep 07 09:29:43.515088 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.515087 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" Sep 07 09:29:43.515316 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.515131 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" event={"ID":"a7da42f5-56c3-4cc6-826c-c558bec8ff5f","Type":"ContainerDied","Data":"846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122"} Sep 07 09:29:43.515316 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.515179 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78" event={"ID":"a7da42f5-56c3-4cc6-826c-c558bec8ff5f","Type":"ContainerDied","Data":"a2aa4f184a96c56c815b0bfe32accb78808b16f74a072eeb3227a96ebd673e4d"} Sep 07 09:29:43.515316 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.515202 2558 scope.go:117] "RemoveContainer" containerID="846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122" Sep 07 09:29:43.525173 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.525154 2558 scope.go:117] "RemoveContainer" containerID="846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122" Sep 07 09:29:43.525437 ip-10-0-131-66 kubenswrapper[2558]: E0907 09:29:43.525419 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122\": container with ID starting with 846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122 not found: ID does not exist" containerID="846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122" Sep 07 09:29:43.525545 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.525449 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122"} err="failed to get container status \"846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122\": rpc error: code = NotFound desc = could not find container \"846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122\": container with ID starting with 846808a477050a9f134944a79ddfb4b1ae50858bf38386497a96efa5abbd1122 not found: ID does not exist" Sep 07 09:29:43.536932 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.536913 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78"] Sep 07 09:29:43.540382 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.540361 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34de95-7454d4d794-t8t78"] Sep 07 09:29:43.952083 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.952050 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7da42f5-56c3-4cc6-826c-c558bec8ff5f" path="/var/lib/kubelet/pods/a7da42f5-56c3-4cc6-826c-c558bec8ff5f/volumes" Sep 07 09:29:43.952404 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:29:43.952391 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f72ba771-294b-4785-a168-b5e9f3b5de03" path="/var/lib/kubelet/pods/f72ba771-294b-4785-a168-b5e9f3b5de03/volumes" Sep 07 09:30:02.363959 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:30:02.363920 2558 scope.go:117] "RemoveContainer" containerID="d63eaf5eee29392499dca1aa7165c69378de43c731e4ac4fadd52d67573f7a34" Sep 07 09:31:02.400307 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:31:02.400273 2558 scope.go:117] "RemoveContainer" containerID="710030094a53be3abc2e34926c67e1fc08e3482e6d376516e97b43c7e2a06bf3" Sep 07 09:31:02.408609 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:31:02.408591 2558 scope.go:117] "RemoveContainer" containerID="c47a1305ba3968812fcca2e1f3bfbff52b928bbd180de876ae555c0262702416" Sep 07 09:31:02.416472 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:31:02.416452 2558 scope.go:117] "RemoveContainer" containerID="aaa7efe99388a1f140fec41f59d4651e62732f976e3fcd67d640814774b688e0" Sep 07 09:34:02.121130 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:34:02.121104 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:34:02.124055 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:34:02.124028 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:34:02.465195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:34:02.465102 2558 scope.go:117] "RemoveContainer" containerID="8823aba76c9b797e9dd674a3e5241603d9f2039c1af05a2047f1d97abd5594ad" Sep 07 09:34:02.473930 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:34:02.473909 2558 scope.go:117] "RemoveContainer" containerID="8d4459800c26d1bdc8d6e1b6576dc10ab0b7950958c0e1c723682408de2b0cf0" Sep 07 09:35:02.498516 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:35:02.498435 2558 scope.go:117] "RemoveContainer" containerID="8920dc8ac0ec708cec12db299ea3ece431a6f3ba51f0538de8892aff9d659e78" Sep 07 09:36:00.634806 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:00.634772 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lkmxx_6f5866f9-d63e-43d6-8cad-1c0e418fd464/server/0.log" Sep 07 09:36:00.747991 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:00.747957 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-s5bdn_11896531-65c0-4f71-98c3-eedd91d81900/manager/0.log" Sep 07 09:36:00.978972 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:00.978899 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-wpx8h_18632cb3-b012-4464-bcc6-73b39d9f6227/manager/0.log" Sep 07 09:36:00.984667 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:00.984642 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-wpx8h_18632cb3-b012-4464-bcc6-73b39d9f6227/copy-manifests/0.log" Sep 07 09:36:02.521355 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:02.521323 2558 scope.go:117] "RemoveContainer" containerID="65a24860a3e2ca4a3bee7aa971afd8cd9846114f6faac3571328c1d09c8ea909" Sep 07 09:36:02.530062 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:02.530043 2558 scope.go:117] "RemoveContainer" containerID="86e308004ae6299c78b4a74c9fc0b5c594b03f5ff57416617df0afb2a5d8c6a3" Sep 07 09:36:03.439348 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:03.439322 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6_ec08351a-7ba8-48b2-b010-a946abe2fa52/istio-proxy/0.log" Sep 07 09:36:03.552272 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:03.552246 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wxd54_b24df2d2-d777-47ba-9509-c3b16745fce7/discovery/0.log" Sep 07 09:36:03.883720 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:03.883690 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck6nc_3b0b6217-00aa-46ef-a29f-55ee000b5152/istio-proxy/1.log" Sep 07 09:36:03.992028 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:03.991990 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q4xpg_8442be03-cf12-40ef-94b2-d45c67ac2e8a/payload-pre-processing/0.log" Sep 07 09:36:04.688636 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:04.688606 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-pndkn_acda191f-8162-43fe-af31-aeaad0d39422/main/0.log" Sep 07 09:36:04.904210 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:04.904184 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-7rmnp_cba918fd-aa93-420a-94af-750652cbe0e0/main/0.log" Sep 07 09:36:05.119372 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:05.119347 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-dg7kx_45c82f05-4450-4d31-a435-f69f9b96f2b5/main/0.log" Sep 07 09:36:12.997665 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:12.997633 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mhtrz_41bba008-7502-4a97-8759-27e21db291c1/global-pull-secret-syncer/0.log" Sep 07 09:36:13.160730 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:13.160699 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ng2zc_035007ac-ac83-4609-8901-0a423a205428/konnectivity-agent/0.log" Sep 07 09:36:13.177191 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:13.177166 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-66.ec2.internal_fba8ac2b9d64ff8f1c61c36855694155/haproxy/0.log" Sep 07 09:36:19.502582 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.502552 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-7ftnw_c0159467-5645-46ef-bc19-96fd5967324d/kube-state-metrics/0.log" Sep 07 09:36:19.523311 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.523280 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-7ftnw_c0159467-5645-46ef-bc19-96fd5967324d/kube-rbac-proxy-main/0.log" Sep 07 09:36:19.549553 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.549524 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-7ftnw_c0159467-5645-46ef-bc19-96fd5967324d/kube-rbac-proxy-self/0.log" Sep 07 09:36:19.734049 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.734019 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hv7zr_755ef248-b213-45b9-aa97-003e40c24356/node-exporter/0.log" Sep 07 09:36:19.750985 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.750960 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hv7zr_755ef248-b213-45b9-aa97-003e40c24356/kube-rbac-proxy/0.log" Sep 07 09:36:19.771533 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.771515 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hv7zr_755ef248-b213-45b9-aa97-003e40c24356/init-textfile/0.log" Sep 07 09:36:19.870381 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.870350 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4wswr_2433411a-91b1-4311-bb6a-51c5aa61d9e2/kube-rbac-proxy-main/0.log" Sep 07 09:36:19.897531 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.897503 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4wswr_2433411a-91b1-4311-bb6a-51c5aa61d9e2/kube-rbac-proxy-self/0.log" Sep 07 09:36:19.921624 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:19.921600 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-4wswr_2433411a-91b1-4311-bb6a-51c5aa61d9e2/openshift-state-metrics/0.log" Sep 07 09:36:20.103864 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:20.103782 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-4c4kb_b029d92e-5d09-4a55-9199-7cb44eb1f552/prometheus-operator/0.log" Sep 07 09:36:20.125402 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:20.125379 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-4c4kb_b029d92e-5d09-4a55-9199-7cb44eb1f552/kube-rbac-proxy/0.log" Sep 07 09:36:20.143460 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:20.143441 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-vvw2w_021ea2e1-192d-45da-9f14-715952579103/prometheus-operator-admission-webhook/0.log" Sep 07 09:36:20.193496 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:20.193468 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f966fffb-9mh48_0f883935-c1c4-48fc-b1ce-048ceb8ef3bf/telemeter-client/0.log" Sep 07 09:36:20.248242 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:20.248211 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f966fffb-9mh48_0f883935-c1c4-48fc-b1ce-048ceb8ef3bf/reload/0.log" Sep 07 09:36:20.273134 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:20.273110 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f966fffb-9mh48_0f883935-c1c4-48fc-b1ce-048ceb8ef3bf/kube-rbac-proxy/0.log" Sep 07 09:36:23.283997 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.283972 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59cccffb88-7q9qc_3f7511e1-8285-4514-bd2a-ba2bf0e4c635/console/0.log" Sep 07 09:36:23.310947 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.310920 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-7dqmf_fdd66e31-1fc9-41c2-8c70-d5dd7c29d836/download-server/0.log" Sep 07 09:36:23.847415 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.847381 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-82228"] Sep 07 09:36:23.847798 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.847749 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f72ba771-294b-4785-a168-b5e9f3b5de03" containerName="revoke-keys" Sep 07 09:36:23.847798 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.847768 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f72ba771-294b-4785-a168-b5e9f3b5de03" containerName="revoke-keys" Sep 07 09:36:23.847972 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.847808 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7da42f5-56c3-4cc6-826c-c558bec8ff5f" containerName="payload-pre-processing" Sep 07 09:36:23.847972 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.847816 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7da42f5-56c3-4cc6-826c-c558bec8ff5f" containerName="payload-pre-processing" Sep 07 09:36:23.847972 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.847932 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f72ba771-294b-4785-a168-b5e9f3b5de03" containerName="revoke-keys" Sep 07 09:36:23.847972 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.847947 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7da42f5-56c3-4cc6-826c-c558bec8ff5f" containerName="payload-pre-processing" Sep 07 09:36:23.851160 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.851142 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:23.853584 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.853554 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5sms\"/\"openshift-service-ca.crt\"" Sep 07 09:36:23.854540 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.854521 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5sms\"/\"kube-root-ca.crt\"" Sep 07 09:36:23.854645 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.854545 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b5sms\"/\"default-dockercfg-xmpz2\"" Sep 07 09:36:23.860815 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:23.860795 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-82228"] Sep 07 09:36:24.014393 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.014357 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-proc\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.014393 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.014397 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-podres\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.014631 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.014456 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xlbx\" (UniqueName: \"kubernetes.io/projected/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-kube-api-access-8xlbx\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.014631 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.014533 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-sys\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.014631 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.014581 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-lib-modules\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115257 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115163 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-lib-modules\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115257 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115243 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-proc\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-podres\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115290 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xlbx\" (UniqueName: \"kubernetes.io/projected/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-kube-api-access-8xlbx\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115360 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-sys\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115373 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-proc\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115387 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-lib-modules\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115412 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-podres\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.115501 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.115434 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-sys\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.122810 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.122783 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xlbx\" (UniqueName: \"kubernetes.io/projected/7fb67c0c-f64f-4e15-ab79-1f00ee58c69c-kube-api-access-8xlbx\") pod \"perf-node-gather-daemonset-82228\" (UID: \"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.160993 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.160969 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.429465 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.429441 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-82228"] Sep 07 09:36:24.431192 ip-10-0-131-66 kubenswrapper[2558]: W0907 09:36:24.431163 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7fb67c0c_f64f_4e15_ab79_1f00ee58c69c.slice/crio-aded9d9a515f4fb89f1ad83208139c5025043168c4d20cf23afda0b424b56862 WatchSource:0}: Error finding container aded9d9a515f4fb89f1ad83208139c5025043168c4d20cf23afda0b424b56862: Status 404 returned error can't find the container with id aded9d9a515f4fb89f1ad83208139c5025043168c4d20cf23afda0b424b56862 Sep 07 09:36:24.750985 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.750901 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" event={"ID":"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c","Type":"ContainerStarted","Data":"ca343c9500dfd93aef510ce5c13fdd6f034d9a0e60525a6c13fdc82d1cc21417"} Sep 07 09:36:24.750985 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.750949 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" event={"ID":"7fb67c0c-f64f-4e15-ab79-1f00ee58c69c","Type":"ContainerStarted","Data":"aded9d9a515f4fb89f1ad83208139c5025043168c4d20cf23afda0b424b56862"} Sep 07 09:36:24.751160 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.751026 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:24.766734 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:24.766687 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" podStartSLOduration=1.766671783 podStartE2EDuration="1.766671783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 09:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:36:24.764400015 +0000 UTC m=+2243.281190917" watchObservedRunningTime="2026-09-07 09:36:24.766671783 +0000 UTC m=+2243.283462686" Sep 07 09:36:25.215030 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:25.214994 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n84s7_783315ec-087a-439f-bad8-138553f6413a/dns/0.log" Sep 07 09:36:25.232755 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:25.232729 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n84s7_783315ec-087a-439f-bad8-138553f6413a/kube-rbac-proxy/0.log" Sep 07 09:36:25.315809 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:25.315782 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtb9n_a55e1e5e-f140-46be-bdb5-1bc383300d9e/dns-node-resolver/0.log" Sep 07 09:36:25.899788 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:25.899751 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f75cx_0cdd0beb-99bd-4921-ba51-e135d1c0a6db/node-ca/0.log" Sep 07 09:36:26.791830 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:26.791803 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6_ec08351a-7ba8-48b2-b010-a946abe2fa52/istio-proxy/0.log" Sep 07 09:36:27.089195 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:27.089109 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wxd54_b24df2d2-d777-47ba-9509-c3b16745fce7/discovery/0.log" Sep 07 09:36:27.153095 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:27.153067 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck6nc_3b0b6217-00aa-46ef-a29f-55ee000b5152/istio-proxy/0.log" Sep 07 09:36:27.163143 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:27.163117 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck6nc_3b0b6217-00aa-46ef-a29f-55ee000b5152/istio-proxy/1.log" Sep 07 09:36:27.194155 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:27.194130 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q4xpg_8442be03-cf12-40ef-94b2-d45c67ac2e8a/payload-pre-processing/0.log" Sep 07 09:36:28.073604 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:28.073577 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-582vt_cde1e62f-e104-437f-82ad-fd68be33a484/serve-healthcheck-canary/0.log" Sep 07 09:36:28.577145 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:28.577112 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j45pg_ab84316f-177d-4c97-b2fd-c630e9a62af2/kube-rbac-proxy/0.log" Sep 07 09:36:28.594833 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:28.594810 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j45pg_ab84316f-177d-4c97-b2fd-c630e9a62af2/exporter/0.log" Sep 07 09:36:28.614144 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:28.614118 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j45pg_ab84316f-177d-4c97-b2fd-c630e9a62af2/extractor/0.log" Sep 07 09:36:30.767032 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:30.767004 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-82228" Sep 07 09:36:31.094757 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:31.094730 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lkmxx_6f5866f9-d63e-43d6-8cad-1c0e418fd464/server/0.log" Sep 07 09:36:31.168740 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:31.168713 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-s5bdn_11896531-65c0-4f71-98c3-eedd91d81900/manager/0.log" Sep 07 09:36:31.230251 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:31.230223 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-wpx8h_18632cb3-b012-4464-bcc6-73b39d9f6227/manager/0.log" Sep 07 09:36:31.246948 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:31.246923 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-wpx8h_18632cb3-b012-4464-bcc6-73b39d9f6227/copy-manifests/0.log" Sep 07 09:36:32.606811 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:32.606781 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-94f7b5c54-xnldj_ff187995-eab2-4c9f-beef-a9e53c21a1bc/manager/0.log" Sep 07 09:36:32.633271 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:32.633247 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8lw4x_77ac6199-037f-43fd-aa42-8e7fdb622b38/openshift-lws-operator/0.log" Sep 07 09:36:33.900355 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:33.900321 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6_ec08351a-7ba8-48b2-b010-a946abe2fa52/istio-proxy/0.log" Sep 07 09:36:34.177760 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:34.177672 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wxd54_b24df2d2-d777-47ba-9509-c3b16745fce7/discovery/0.log" Sep 07 09:36:34.222484 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:34.222451 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck6nc_3b0b6217-00aa-46ef-a29f-55ee000b5152/istio-proxy/0.log" Sep 07 09:36:34.232260 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:34.232240 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck6nc_3b0b6217-00aa-46ef-a29f-55ee000b5152/istio-proxy/1.log" Sep 07 09:36:34.253343 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:34.253322 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q4xpg_8442be03-cf12-40ef-94b2-d45c67ac2e8a/payload-pre-processing/0.log" Sep 07 09:36:35.848177 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:35.848139 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-87gw6_ec08351a-7ba8-48b2-b010-a946abe2fa52/istio-proxy/0.log" Sep 07 09:36:36.131592 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:36.131511 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-wxd54_b24df2d2-d777-47ba-9509-c3b16745fce7/discovery/0.log" Sep 07 09:36:36.179190 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:36.179160 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck6nc_3b0b6217-00aa-46ef-a29f-55ee000b5152/istio-proxy/0.log" Sep 07 09:36:36.188469 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:36.188444 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ck6nc_3b0b6217-00aa-46ef-a29f-55ee000b5152/istio-proxy/1.log" Sep 07 09:36:36.212431 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:36.212404 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-q4xpg_8442be03-cf12-40ef-94b2-d45c67ac2e8a/payload-pre-processing/0.log" Sep 07 09:36:39.045497 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.045463 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjtz8_bdb82087-8753-4795-80ff-c9eae563fcc3/kube-multus-additional-cni-plugins/0.log" Sep 07 09:36:39.064270 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.064245 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjtz8_bdb82087-8753-4795-80ff-c9eae563fcc3/egress-router-binary-copy/0.log" Sep 07 09:36:39.081341 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.081319 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjtz8_bdb82087-8753-4795-80ff-c9eae563fcc3/cni-plugins/0.log" Sep 07 09:36:39.101273 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.101250 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjtz8_bdb82087-8753-4795-80ff-c9eae563fcc3/bond-cni-plugin/0.log" Sep 07 09:36:39.118463 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.118443 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjtz8_bdb82087-8753-4795-80ff-c9eae563fcc3/routeoverride-cni/0.log" Sep 07 09:36:39.135258 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.135238 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjtz8_bdb82087-8753-4795-80ff-c9eae563fcc3/whereabouts-cni-bincopy/0.log" Sep 07 09:36:39.153095 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.153076 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjtz8_bdb82087-8753-4795-80ff-c9eae563fcc3/whereabouts-cni/0.log" Sep 07 09:36:39.380285 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.380210 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wf8cn_0e187155-3f93-4931-ba83-8f896446b9c2/kube-multus/0.log" Sep 07 09:36:39.473686 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.473657 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nzmrx_3ce552e2-a957-4a53-b836-e8ffec7bfb37/network-metrics-daemon/0.log" Sep 07 09:36:39.488607 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:39.488580 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nzmrx_3ce552e2-a957-4a53-b836-e8ffec7bfb37/kube-rbac-proxy/0.log" Sep 07 09:36:40.229946 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.229918 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-controller/0.log" Sep 07 09:36:40.243586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.243558 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/1.log" Sep 07 09:36:40.243586 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.243574 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovn-acl-logging/0.log" Sep 07 09:36:40.259158 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.259139 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/kube-rbac-proxy-node/0.log" Sep 07 09:36:40.277783 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.277764 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/kube-rbac-proxy-ovn-metrics/0.log" Sep 07 09:36:40.294915 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.294900 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/northd/0.log" Sep 07 09:36:40.311831 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.311811 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/nbdb/0.log" Sep 07 09:36:40.329551 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.329528 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/sbdb/0.log" Sep 07 09:36:40.380155 ip-10-0-131-66 kubenswrapper[2558]: I0907 09:36:40.380131 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5gdns_1369d2eb-8b75-4061-9769-b75f5e840e21/ovnkube-controller/0.log"