Sep 19 09:04:44.660260 ip-10-0-132-57 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 19 09:04:44.660274 ip-10-0-132-57 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 19 09:04:44.660283 ip-10-0-132-57 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 19 09:04:44.660593 ip-10-0-132-57 systemd[1]: Failed to start Kubernetes Kubelet. Sep 19 09:04:53.756041 ip-10-0-132-57 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 19 09:04:53.756057 ip-10-0-132-57 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 17142321c35f49688395ebb8409bc034 -- Sep 19 09:06:31.617987 ip-10-0-132-57 systemd[1]: Starting Kubernetes Kubelet... Sep 19 09:06:35.265174 ip-10-0-132-57 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 09:06:35.265174 ip-10-0-132-57 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 19 09:06:35.265174 ip-10-0-132-57 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 09:06:35.265174 ip-10-0-132-57 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 19 09:06:35.265174 ip-10-0-132-57 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 09:06:35.266135 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.265260 2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 19 09:06:35.268887 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268864 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:06:35.268887 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268886 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:06:35.268887 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268890 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268893 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268897 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268900 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268903 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268908 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268912 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268916 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268919 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268922 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268931 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268934 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268937 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268940 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268942 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268946 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268948 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268951 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268954 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:06:35.268994 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268957 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268960 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268962 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268965 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268968 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268971 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268974 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268976 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268980 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268983 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268986 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268988 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268991 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268993 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268996 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.268999 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269002 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269005 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269008 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269011 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:06:35.269453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269013 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269016 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269018 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269020 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269023 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269026 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269028 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269031 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269033 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269035 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269037 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269040 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269042 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269045 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269049 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269052 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269054 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269057 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269061 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269064 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:06:35.270003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269067 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269070 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269073 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269076 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269079 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269082 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269084 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269087 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269090 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269092 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269095 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269098 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269100 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269102 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269105 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269107 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269110 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269112 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269116 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:06:35.270597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269118 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269121 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269123 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269127 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269131 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269133 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269136 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269139 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269142 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269145 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269148 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269150 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269153 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269155 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269158 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269160 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269163 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269166 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269168 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:06:35.271058 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269895 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269902 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269905 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269908 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269911 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269914 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269917 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269920 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269922 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269925 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269928 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269931 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269933 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269936 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269939 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269942 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269944 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269947 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269950 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:06:35.271535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269952 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269958 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269960 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269963 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269966 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269969 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269971 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269974 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269977 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269979 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269982 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269985 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269988 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269990 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269993 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269996 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.269999 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270001 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270004 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270007 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:06:35.272032 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270010 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270013 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270019 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270023 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270026 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270029 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270032 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270035 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270038 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270040 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270043 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270045 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270048 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270052 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270055 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270058 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270061 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270064 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270067 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:06:35.272535 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270070 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270072 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270075 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270077 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270080 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270083 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270086 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270088 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270091 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270093 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270097 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270102 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270105 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270109 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270112 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270115 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270118 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270120 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270123 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:06:35.273021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270125 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270128 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270130 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270133 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270135 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270138 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270140 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270143 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270146 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270149 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270152 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270155 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270157 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270160 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270162 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270165 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270168 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270170 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270173 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270175 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:06:35.273549 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270178 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270181 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270640 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270647 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270650 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270653 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270657 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270660 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270663 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270666 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270669 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270672 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270675 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270677 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270680 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270683 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270685 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270688 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270691 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270694 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:06:35.274078 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270696 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270699 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270702 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270704 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270707 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270710 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270712 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270715 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270718 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270720 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270723 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270725 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270728 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270730 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270733 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270736 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270738 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270740 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270743 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270747 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:06:35.274586 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270750 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270752 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270755 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270757 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270761 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270764 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270766 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270769 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270771 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270774 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270776 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270782 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270785 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270787 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270790 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270792 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270795 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270797 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270800 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270802 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:06:35.275057 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270805 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270807 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270810 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270812 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270815 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270817 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270820 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270822 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270826 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270830 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270832 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270835 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270838 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270840 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270843 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270846 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270849 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270852 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270854 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:06:35.275595 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270857 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270859 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270863 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270865 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270870 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270874 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270876 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270879 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270881 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270884 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270887 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270890 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270892 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270895 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270897 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270900 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270902 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270904 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270907 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270910 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:06:35.276033 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270912 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.270917 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271002 2585 flags.go:64] FLAG: --address="0.0.0.0" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271047 2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271055 2585 flags.go:64] FLAG: --anonymous-auth="true" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271059 2585 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271065 2585 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271068 2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271073 2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271078 2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271082 2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271085 2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271089 2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271092 2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271095 2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271098 2585 flags.go:64] FLAG: --cgroup-root="" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271103 2585 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271106 2585 flags.go:64] FLAG: --client-ca-file="" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271109 2585 flags.go:64] FLAG: --cloud-provider="external" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271112 2585 flags.go:64] FLAG: --cluster-dns="[]" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271117 2585 flags.go:64] FLAG: --cluster-domain="" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271120 2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271123 2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 19 09:06:35.276567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271126 2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271129 2585 flags.go:64] FLAG: --container-log-max-files="5" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271133 2585 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271136 2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271140 2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271143 2585 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271147 2585 flags.go:64] FLAG: --contention-profiling="false" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271150 2585 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271153 2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271156 2585 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271159 2585 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271164 2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271167 2585 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271170 2585 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271172 2585 flags.go:64] FLAG: --enable-load-reader="false" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271176 2585 flags.go:64] FLAG: --enable-server="true" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271180 2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271204 2585 flags.go:64] FLAG: --event-burst="100" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271208 2585 flags.go:64] FLAG: --event-qps="50" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271211 2585 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271215 2585 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271218 2585 flags.go:64] FLAG: --eviction-hard="" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271222 2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271225 2585 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271228 2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 19 09:06:35.277120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271232 2585 flags.go:64] FLAG: --eviction-soft="" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271235 2585 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271238 2585 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271241 2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271244 2585 flags.go:64] FLAG: --experimental-mounter-path="" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271247 2585 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271250 2585 flags.go:64] FLAG: --fail-swap-on="true" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271253 2585 flags.go:64] FLAG: --feature-gates="" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271257 2585 flags.go:64] FLAG: --file-check-frequency="20s" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271260 2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271263 2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271267 2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271269 2585 flags.go:64] FLAG: --healthz-port="10248" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271272 2585 flags.go:64] FLAG: --help="false" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271276 2585 flags.go:64] FLAG: --hostname-override="ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271278 2585 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271281 2585 flags.go:64] FLAG: --http-check-frequency="20s" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271285 2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271288 2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271292 2585 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271295 2585 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271297 2585 flags.go:64] FLAG: --image-service-endpoint="" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271301 2585 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 19 09:06:35.277731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271303 2585 flags.go:64] FLAG: --kube-api-burst="100" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271307 2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271310 2585 flags.go:64] FLAG: --kube-api-qps="50" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271313 2585 flags.go:64] FLAG: --kube-reserved="" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271316 2585 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271319 2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271322 2585 flags.go:64] FLAG: --kubelet-cgroups="" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271325 2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271328 2585 flags.go:64] FLAG: --lock-file="" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271332 2585 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271335 2585 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271338 2585 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271344 2585 flags.go:64] FLAG: --log-json-split-stream="false" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271347 2585 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271349 2585 flags.go:64] FLAG: --log-text-split-stream="false" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271352 2585 flags.go:64] FLAG: --logging-format="text" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271355 2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271358 2585 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271362 2585 flags.go:64] FLAG: --manifest-url="" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271365 2585 flags.go:64] FLAG: --manifest-url-header="" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271369 2585 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271372 2585 flags.go:64] FLAG: --max-open-files="1000000" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271377 2585 flags.go:64] FLAG: --max-pods="110" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271379 2585 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271383 2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 19 09:06:35.278297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271386 2585 flags.go:64] FLAG: --memory-manager-policy="None" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271391 2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271395 2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271398 2585 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271401 2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271409 2585 flags.go:64] FLAG: --node-status-max-images="50" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271412 2585 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271414 2585 flags.go:64] FLAG: --oom-score-adj="-999" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271417 2585 flags.go:64] FLAG: --pod-cidr="" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271421 2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271428 2585 flags.go:64] FLAG: --pod-manifest-path="" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271431 2585 flags.go:64] FLAG: --pod-max-pids="-1" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271434 2585 flags.go:64] FLAG: --pods-per-core="0" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271437 2585 flags.go:64] FLAG: --port="10250" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271440 2585 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271443 2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-049b2379d085c5695" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271446 2585 flags.go:64] FLAG: --qos-reserved="" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271450 2585 flags.go:64] FLAG: --read-only-port="10255" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271453 2585 flags.go:64] FLAG: --register-node="true" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271457 2585 flags.go:64] FLAG: --register-with-taints="" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271460 2585 flags.go:64] FLAG: --registry-burst="10" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271463 2585 flags.go:64] FLAG: --registry-qps="5" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271466 2585 flags.go:64] FLAG: --reserved-cpus="" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271469 2585 flags.go:64] FLAG: --reserved-memory="" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271473 2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 19 09:06:35.278939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271476 2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271479 2585 flags.go:64] FLAG: --rotate-certificates="false" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271482 2585 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271485 2585 flags.go:64] FLAG: --runonce="false" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271488 2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271491 2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271494 2585 flags.go:64] FLAG: --seccomp-default="false" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271497 2585 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271501 2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271505 2585 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271508 2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271511 2585 flags.go:64] FLAG: --storage-driver-password="root" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271529 2585 flags.go:64] FLAG: --storage-driver-secure="false" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271532 2585 flags.go:64] FLAG: --storage-driver-table="stats" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271535 2585 flags.go:64] FLAG: --storage-driver-user="root" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271538 2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271541 2585 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271545 2585 flags.go:64] FLAG: --system-cgroups="" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271548 2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271553 2585 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271556 2585 flags.go:64] FLAG: --tls-cert-file="" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271560 2585 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271564 2585 flags.go:64] FLAG: --tls-min-version="" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271567 2585 flags.go:64] FLAG: --tls-private-key-file="" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271571 2585 flags.go:64] FLAG: --topology-manager-policy="none" Sep 19 09:06:35.279791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271574 2585 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271577 2585 flags.go:64] FLAG: --topology-manager-scope="container" Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271580 2585 flags.go:64] FLAG: --v="2" Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271584 2585 flags.go:64] FLAG: --version="false" Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271588 2585 flags.go:64] FLAG: --vmodule="" Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271592 2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.271595 2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271721 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271725 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271729 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271732 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271735 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271737 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271740 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271743 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271751 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271754 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271756 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271759 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271762 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271764 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271767 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:06:35.280731 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271770 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271773 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271775 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271778 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271781 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271784 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271786 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271789 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271791 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271795 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271798 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271801 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271805 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271809 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271812 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271815 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271818 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271821 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271823 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271826 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:06:35.281270 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271829 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271831 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271834 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271836 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271839 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271843 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271846 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271848 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271851 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271853 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271856 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271858 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271861 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271864 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271866 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271869 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271872 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271876 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271880 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271883 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:06:35.281982 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271886 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271889 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271892 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271894 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271897 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271900 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271903 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271905 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271908 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271910 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271912 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271915 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271917 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271920 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271922 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271925 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271927 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271932 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271935 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271937 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:06:35.282476 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271940 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271942 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271945 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271948 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271950 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271953 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271955 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271958 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271960 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271963 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271966 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271969 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271971 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271974 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271977 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271979 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271982 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271984 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271987 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271989 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:06:35.282980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271992 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271994 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.271997 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.272001 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.272556 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.281875 2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.281896 2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.281918 2585 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.281924 2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.281986 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.281991 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.281996 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282000 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:06:35.283475 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282004 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282008 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282011 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282014 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282017 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282020 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282023 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282026 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282029 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282032 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282036 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282038 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282041 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282045 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282047 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282050 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282053 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282055 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282058 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:06:35.283827 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282061 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282063 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282066 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282069 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282072 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282075 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282078 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282080 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282083 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282086 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282088 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282091 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282094 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282097 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282100 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282102 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282105 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282108 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282110 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282113 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:06:35.284312 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282116 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282118 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282120 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282123 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282125 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282128 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282131 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282133 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282136 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282139 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282142 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282145 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282147 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282149 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282152 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282154 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282157 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282159 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282162 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282164 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:06:35.284844 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282168 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282172 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282175 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282178 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282180 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282183 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282185 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282190 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282193 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282196 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282198 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282201 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282204 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282206 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282209 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282211 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282215 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282217 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282220 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282223 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:06:35.285378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282225 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282228 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282230 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282233 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282236 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282239 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282241 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282244 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282246 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282249 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282252 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282254 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282257 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282260 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282263 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:06:35.285896 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282266 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.282271 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282404 2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282409 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282412 2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282415 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282418 2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282421 2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282423 2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282426 2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282429 2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282431 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282434 2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282436 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282440 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:06:35.286274 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282443 2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282446 2585 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282448 2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282451 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282453 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282456 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282459 2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282462 2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282465 2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282467 2585 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282470 2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282473 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282475 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282478 2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282481 2585 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282484 2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282487 2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282490 2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282493 2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282496 2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:06:35.287092 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282498 2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282501 2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282504 2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282507 2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282509 2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282512 2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282529 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282534 2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282537 2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282540 2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282542 2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282545 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282550 2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282553 2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282556 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282559 2585 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282562 2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282565 2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282567 2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282570 2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:06:35.287802 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282572 2585 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282575 2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282578 2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282582 2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282586 2585 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282588 2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282591 2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282594 2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282597 2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282600 2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282603 2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282605 2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282608 2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282611 2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282613 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282616 2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282619 2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282623 2585 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282626 2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:06:35.288290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282629 2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282632 2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282636 2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282639 2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282642 2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282645 2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282652 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282655 2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282658 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282661 2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282664 2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282667 2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282669 2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282672 2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282675 2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282677 2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282680 2585 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282683 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282686 2585 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:06:35.288893 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282688 2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282691 2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282694 2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282696 2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282699 2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282701 2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282704 2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:35.282707 2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.282711 2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.282855 2585 server.go:962] "Client rotation is on, will bootstrap in background" Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.284875 2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.286266 2585 server.go:1019] "Starting client certificate rotation" Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.286389 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 09:06:35.289433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.286446 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 09:06:35.289783 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.288895 2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 09:06:35.290160 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.290137 2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 09:06:35.295241 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.295218 2585 log.go:25] "Validated CRI v1 runtime API" Sep 19 09:06:35.296919 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.296898 2585 log.go:25] "Validated CRI v1 image API" Sep 19 09:06:35.298304 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.298287 2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 19 09:06:35.302956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.302929 2585 fs.go:135] Filesystem UUIDs: map[2be3920c-6e4e-46bf-97de-c8d85408d750:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 804b8f25-3d1c-4cc7-8e96-c0cc0496880e:/dev/nvme0n1p3] Sep 19 09:06:35.303066 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.302951 2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 19 09:06:35.308891 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.308745 2585 manager.go:216] Machine: {Timestamp:2026-09-19 09:06:35.3082287 +0000 UTC m=+2.066349716 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3174690 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec291058d90354700f598dcba89fc8bb SystemUUID:ec291058-d903-5470-0f59-8dcba89fc8bb BootID:17142321-c35f-4968-8395-ebb8409bc034 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e8:4f:0f:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e8:4f:0f:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:07:35:b0:2b:5a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 19 09:06:35.308891 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.308865 2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 19 09:06:35.309069 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.308967 2585 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 19 09:06:35.310543 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.310488 2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 19 09:06:35.310693 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.310544 2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-57.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 19 09:06:35.311612 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.311598 2585 topology_manager.go:138] "Creating topology manager with none policy" Sep 19 09:06:35.311647 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.311615 2585 container_manager_linux.go:309] "Creating device plugin manager" Sep 19 09:06:35.311647 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.311624 2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 09:06:35.311647 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.311637 2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 09:06:35.312702 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.312688 2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 19 09:06:35.316227 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316211 2585 state_mem.go:36] "Initialized new in-memory state store" Sep 19 09:06:35.316410 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316401 2585 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 19 09:06:35.316442 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316435 2585 kubelet.go:486] "Attempting to sync node with API server" Sep 19 09:06:35.316469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316445 2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 19 09:06:35.316469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316461 2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 19 09:06:35.316540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316470 2585 kubelet.go:398] "Adding apiserver pod source" Sep 19 09:06:35.316540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316479 2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 19 09:06:35.316848 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316834 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 09:06:35.316896 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.316854 2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 09:06:35.319129 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.319108 2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 19 09:06:35.321565 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321506 2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 19 09:06:35.321666 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321571 2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 19 09:06:35.321782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321733 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 09:06:35.321843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321783 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 19 09:06:35.321843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321802 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 19 09:06:35.321843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321811 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 19 09:06:35.321843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321821 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 19 09:06:35.321843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321834 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 19 09:06:35.321843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.321842 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 19 09:06:35.322436 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322425 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 19 09:06:35.322489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322437 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 19 09:06:35.322489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322449 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 19 09:06:35.322489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322458 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 19 09:06:35.322489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322470 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 19 09:06:35.322489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322482 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 19 09:06:35.322675 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322532 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 19 09:06:35.322675 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.322541 2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 19 09:06:35.326151 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.326114 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 19 09:06:35.326275 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.326168 2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-57.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 19 09:06:35.326347 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.326296 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 19 09:06:35.329691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.326661 2585 server.go:1268] "Started kubelet" Sep 19 09:06:35.329691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.326706 2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 19 09:06:35.329691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.326729 2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 19 09:06:35.329691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.326787 2585 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 19 09:06:35.329691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.327508 2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 19 09:06:35.329691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.328535 2585 server.go:310] "Adding debug handlers to kubelet server" Sep 19 09:06:35.327546 ip-10-0-132-57 systemd[1]: Started Kubernetes Kubelet. Sep 19 09:06:35.356278 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.356241 2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 19 09:06:35.358181 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.358159 2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 19 09:06:35.358181 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.358177 2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 19 09:06:35.358401 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.358381 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:35.358858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.358827 2585 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 19 09:06:35.358858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.358847 2585 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 19 09:06:35.359034 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.358830 2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 19 09:06:35.359034 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.358981 2585 reconstruct.go:97] "Volume reconstruction finished" Sep 19 09:06:35.359034 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.358989 2585 reconciler.go:29] "Reconciler: start to sync state" Sep 19 09:06:35.359372 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.359352 2585 factory.go:55] Registering systemd factory Sep 19 09:06:35.359372 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.359374 2585 factory.go:223] Registration of the systemd container factory successfully Sep 19 09:06:35.360146 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.359139 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-57.ec2.internal.18d6ad9dd5503f0b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-57.ec2.internal,UID:ip-10-0-132-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-57.ec2.internal,},FirstTimestamp:2026-09-19 09:06:35.326627595 +0000 UTC m=+2.084748667,LastTimestamp:2026-09-19 09:06:35.326627595 +0000 UTC m=+2.084748667,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-57.ec2.internal,}" Sep 19 09:06:35.361599 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.361582 2585 factory.go:153] Registering CRI-O factory Sep 19 09:06:35.361599 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.361601 2585 factory.go:223] Registration of the crio container factory successfully Sep 19 09:06:35.361711 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.361658 2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 19 09:06:35.361711 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.361679 2585 factory.go:103] Registering Raw factory Sep 19 09:06:35.361711 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.361695 2585 manager.go:1196] Started watching for new ooms in manager Sep 19 09:06:35.362257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.362235 2585 manager.go:349] Starting recovery of all containers Sep 19 09:06:35.365697 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.365666 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vz6cn" Sep 19 09:06:35.365827 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.365735 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 19 09:06:35.367949 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.367912 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 19 09:06:35.368118 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.368094 2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 19 09:06:35.372901 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.372861 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vz6cn" Sep 19 09:06:35.373650 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.373633 2585 manager.go:354] Recovery completed Sep 19 09:06:35.378665 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.378495 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:06:35.380889 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.380867 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:06:35.380984 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.380901 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:06:35.380984 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.380919 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:06:35.381514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.381498 2585 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 19 09:06:35.381514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.381512 2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 19 09:06:35.381614 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.381513 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:06:35.381614 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.381545 2585 state_mem.go:36] "Initialized new in-memory state store" Sep 19 09:06:35.382951 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.382858 2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-57.ec2.internal.18d6ad9dd88c3088 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-57.ec2.internal,UID:ip-10-0-132-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-57.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-57.ec2.internal,},FirstTimestamp:2026-09-19 09:06:35.380887688 +0000 UTC m=+2.139008706,LastTimestamp:2026-09-19 09:06:35.380887688 +0000 UTC m=+2.139008706,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-57.ec2.internal,}" Sep 19 09:06:35.383232 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.383216 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:06:35.383284 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.383245 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:06:35.383284 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.383255 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:06:35.383538 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.383513 2585 policy_none.go:49] "None policy: Start" Sep 19 09:06:35.383578 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.383542 2585 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 19 09:06:35.383578 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.383553 2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 19 09:06:35.385488 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.385473 2585 policy_none.go:47] "Start" Sep 19 09:06:35.429283 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.429252 2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 19 09:06:35.432935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.429310 2585 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 19 09:06:35.432935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.429333 2585 kubelet.go:2448] "Starting kubelet main sync loop" Sep 19 09:06:35.432935 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.429383 2585 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Sep 19 09:06:35.433531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.433499 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:06:35.445306 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.445284 2585 manager.go:337] "Starting Device Plugin manager" Sep 19 09:06:35.445444 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.445330 2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 19 09:06:35.445444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.445341 2585 server.go:87] "Starting device plugin registration server" Sep 19 09:06:35.445704 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.445689 2585 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 19 09:06:35.445786 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.445715 2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 19 09:06:35.445835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.445790 2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 19 09:06:35.445887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.445878 2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 19 09:06:35.445887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.445887 2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 19 09:06:35.448333 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.447881 2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 19 09:06:35.448333 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.448163 2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:35.530472 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.529823 2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal"] Sep 19 09:06:35.530472 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.530469 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:06:35.531620 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.531599 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:06:35.531734 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.531632 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:06:35.531734 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.531648 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:06:35.533907 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.533888 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:06:35.534016 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534000 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.534069 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534034 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:06:35.534829 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534808 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:06:35.534917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534832 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:06:35.534917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534848 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:06:35.534917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534854 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:06:35.534917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534863 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:06:35.534917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.534869 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:06:35.537101 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.537082 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.537179 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.537120 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:06:35.538042 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.538021 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:06:35.538147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.538057 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:06:35.538147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.538071 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:06:35.545937 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.545911 2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:06:35.546932 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.546912 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:06:35.547055 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.546942 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:06:35.547055 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.546954 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:06:35.547055 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.546977 2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.555790 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.555761 2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.555887 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.555799 2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-57.ec2.internal\": node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:35.559632 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.559603 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-57.ec2.internal\" not found" node="ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.564388 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.564367 2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-57.ec2.internal\" not found" node="ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.575710 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.575682 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:35.660566 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.660509 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d60026aede2bdb637006ae8e684c1f0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"5d60026aede2bdb637006ae8e684c1f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.660566 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.660560 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d60026aede2bdb637006ae8e684c1f0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"5d60026aede2bdb637006ae8e684c1f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.660785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.660580 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03b66c5394879f7702479f341fee142c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-57.ec2.internal\" (UID: \"03b66c5394879f7702479f341fee142c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.676621 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.676581 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:35.761269 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.761233 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d60026aede2bdb637006ae8e684c1f0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"5d60026aede2bdb637006ae8e684c1f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.761447 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.761277 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d60026aede2bdb637006ae8e684c1f0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"5d60026aede2bdb637006ae8e684c1f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.761447 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.761306 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03b66c5394879f7702479f341fee142c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-57.ec2.internal\" (UID: \"03b66c5394879f7702479f341fee142c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.761447 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.761334 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d60026aede2bdb637006ae8e684c1f0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"5d60026aede2bdb637006ae8e684c1f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.761447 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.761352 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/03b66c5394879f7702479f341fee142c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-57.ec2.internal\" (UID: \"03b66c5394879f7702479f341fee142c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.761447 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.761334 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5d60026aede2bdb637006ae8e684c1f0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal\" (UID: \"5d60026aede2bdb637006ae8e684c1f0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.777378 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.777343 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:35.862772 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.862726 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.867970 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:35.867790 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Sep 19 09:06:35.877660 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.877634 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:35.978306 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:35.978261 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.078839 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:36.078803 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.179452 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:36.179359 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.265892 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.265861 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:06:36.280235 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:36.280201 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.285386 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.285359 2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 19 09:06:36.285563 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.285540 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 19 09:06:36.285606 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.285569 2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 19 09:06:36.377694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.377640 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 09:01:35 +0000 UTC" deadline="2028-06-09 18:26:35.949038974 +0000 UTC" Sep 19 09:06:36.377694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.377682 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15105h19m59.571360308s" Sep 19 09:06:36.380833 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:36.380806 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.481685 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:36.481649 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.511981 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:36.511940 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d60026aede2bdb637006ae8e684c1f0.slice/crio-0c1000653eb3e5bb28ef3a4e4301b0ce14048e06283352885427cebcb7342222 WatchSource:0}: Error finding container 0c1000653eb3e5bb28ef3a4e4301b0ce14048e06283352885427cebcb7342222: Status 404 returned error can't find the container with id 0c1000653eb3e5bb28ef3a4e4301b0ce14048e06283352885427cebcb7342222 Sep 19 09:06:36.512423 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:36.512400 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b66c5394879f7702479f341fee142c.slice/crio-045acf569aab038bfad597f54b085be432ad965dbb0f8327b181d3ddf39067bf WatchSource:0}: Error finding container 045acf569aab038bfad597f54b085be432ad965dbb0f8327b181d3ddf39067bf: Status 404 returned error can't find the container with id 045acf569aab038bfad597f54b085be432ad965dbb0f8327b181d3ddf39067bf Sep 19 09:06:36.517879 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.517860 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:06:36.582291 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:36.582248 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.682880 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:36.682833 2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-57.ec2.internal\" not found" Sep 19 09:06:36.726832 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.726752 2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:06:36.765205 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.765169 2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:06:36.859113 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.859077 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" Sep 19 09:06:36.871625 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.871594 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 09:06:36.872594 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.872575 2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" Sep 19 09:06:36.882412 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:36.882383 2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 09:06:37.318542 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.318346 2585 apiserver.go:52] "Watching apiserver" Sep 19 09:06:37.325327 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.325293 2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 19 09:06:37.325896 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.325871 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal","openshift-multus/multus-additional-cni-plugins-chtr4","kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal","openshift-image-registry/node-ca-kghq6","openshift-multus/multus-5jch7","openshift-multus/network-metrics-daemon-nst5s","openshift-network-diagnostics/network-check-target-gml7f","openshift-network-operator/iptables-alerter-tqpch","openshift-ovn-kubernetes/ovnkube-node-6c442","kube-system/global-pull-secret-syncer-fljvj","kube-system/konnectivity-agent-lrw5b","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9","openshift-cluster-node-tuning-operator/tuned-9nfpt","openshift-dns/node-resolver-wcqst"] Sep 19 09:06:37.328498 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.328474 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:37.328640 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.328577 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:37.331058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.330812 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.333120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.333092 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.335753 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.334457 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 19 09:06:37.335753 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.334708 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tqk6r\"" Sep 19 09:06:37.335753 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.334751 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:06:37.335753 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.335689 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5sslj\"" Sep 19 09:06:37.335951 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.335786 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 19 09:06:37.336330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.336313 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 19 09:06:37.337718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.337695 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.339833 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.339808 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.339996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.339973 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nvxjc\"" Sep 19 09:06:37.340094 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.340058 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 19 09:06:37.341012 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.340872 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 19 09:06:37.341114 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.341044 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 19 09:06:37.341982 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.341853 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 19 09:06:37.342188 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.342167 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 19 09:06:37.342543 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.342419 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 19 09:06:37.342633 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.342616 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 19 09:06:37.342770 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.342623 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-st6b2\"" Sep 19 09:06:37.343411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.343133 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:37.343411 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.343210 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:37.346211 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.346191 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.348401 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.348380 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.350903 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.350877 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 19 09:06:37.350996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.350934 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:06:37.351059 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.350993 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cl85h\"" Sep 19 09:06:37.351059 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.351017 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 19 09:06:37.351059 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.350931 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 19 09:06:37.351199 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.350888 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 19 09:06:37.351199 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.351151 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 19 09:06:37.351664 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.351392 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.351664 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.351546 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 19 09:06:37.351933 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.351916 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 19 09:06:37.352118 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.352105 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xcghg\"" Sep 19 09:06:37.352338 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.352322 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 19 09:06:37.353736 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.353715 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.353827 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.353804 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:37.354620 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.354602 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 19 09:06:37.354708 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.354608 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fs6mh\"" Sep 19 09:06:37.354831 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.354814 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 19 09:06:37.357960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.357933 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.358615 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.358583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.359623 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.359605 2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 19 09:06:37.360010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.359993 2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 19 09:06:37.361061 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.360741 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 19 09:06:37.361061 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.360771 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 19 09:06:37.361878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.361859 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 19 09:06:37.362111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.362095 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mkfsk\"" Sep 19 09:06:37.362279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.362169 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cmxbx\"" Sep 19 09:06:37.362279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.362265 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 19 09:06:37.362391 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.362368 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-ovnkube-script-lib\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371511 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-daemon-config\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371553 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371589 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-sys-fs\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371611 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-sys\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371659 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5c5af80-f492-4408-aa6f-310bf3fac70d-host-slash\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371727 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1e4bdae-16ac-478a-be9d-b108434edc8b-serviceca\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371785 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-cnibin\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371822 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-socket-dir-parent\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371856 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-netns\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371892 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvtwf\" (UniqueName: \"kubernetes.io/projected/71b11675-036f-4c23-a38e-f617f769ea69-kube-api-access-bvtwf\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371932 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-kubernetes\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.371970 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-systemd\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372003 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-var-lib-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372025 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-ovnkube-config\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.372388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372046 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fe574be-a073-4960-803c-3d65227fc764-ovn-node-metrics-cert\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372098 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5hvl\" (UniqueName: \"kubernetes.io/projected/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-kube-api-access-x5hvl\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372120 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-device-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372141 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372162 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysctl-d\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372218 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-run-netns\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372245 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8jfg\" (UniqueName: \"kubernetes.io/projected/c5c5af80-f492-4408-aa6f-310bf3fac70d-kube-api-access-m8jfg\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372270 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372292 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-system-cni-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372386 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjs76\" (UniqueName: \"kubernetes.io/projected/76ea5338-fc9a-45a1-9111-a727e59130fa-kube-api-access-kjs76\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372429 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372507 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-conf-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372551 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-lib-modules\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372605 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-tmp\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372637 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-kubelet\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372668 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-systemd\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372705 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxkcz\" (UniqueName: \"kubernetes.io/projected/4fe574be-a073-4960-803c-3d65227fc764-kube-api-access-wxkcz\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372734 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372758 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-run\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372792 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-log-socket\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372823 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372849 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6661651c-f4e1-4a3b-9b68-310a36c28f6b-konnectivity-ca\") pod \"konnectivity-agent-lrw5b\" (UID: \"6661651c-f4e1-4a3b-9b68-310a36c28f6b\") " pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372871 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372892 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1e4bdae-16ac-478a-be9d-b108434edc8b-host\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372934 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-systemd-units\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372965 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c5c5af80-f492-4408-aa6f-310bf3fac70d-iptables-alerter-script\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.372994 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-k8s-cni-cncf-io\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373020 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-etc-kubernetes\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373048 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysctl-conf\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373077 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9szdh\" (UniqueName: \"kubernetes.io/projected/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-kube-api-access-9szdh\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373103 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlnk8\" (UniqueName: \"kubernetes.io/projected/2f96adac-d4cf-4684-b8cf-2a87800cbf10-kube-api-access-tlnk8\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373128 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-system-cni-dir\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.374855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373149 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373174 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkv6f\" (UniqueName: \"kubernetes.io/projected/c1e4bdae-16ac-478a-be9d-b108434edc8b-kube-api-access-gkv6f\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373205 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-cni-bin\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373227 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-cni-multus\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373258 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f96adac-d4cf-4684-b8cf-2a87800cbf10-hosts-file\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373290 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f96adac-d4cf-4684-b8cf-2a87800cbf10-tmp-dir\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373337 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-kubelet-config\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373360 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-cni-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373387 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-multus-certs\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373413 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373443 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-registration-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373469 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6661651c-f4e1-4a3b-9b68-310a36c28f6b-agent-certs\") pod \"konnectivity-agent-lrw5b\" (UID: \"6661651c-f4e1-4a3b-9b68-310a36c28f6b\") " pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373546 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01b00b26-83e4-49bd-bc69-04501dadedd4-cni-binary-copy\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373585 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysconfig\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373615 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-var-lib-kubelet\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373643 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-etc-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373681 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-cni-netd\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.375474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373709 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-hostroot\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373730 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-host\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373771 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373799 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-node-log\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373821 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-run-ovn-kubernetes\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373853 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-env-overrides\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373879 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-kubelet\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373929 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-cni-bin\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373960 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.373988 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-socket-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.374010 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-modprobe-d\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.374033 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-tuned\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.374058 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-slash\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.374085 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-ovn\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.374107 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-cnibin\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.374132 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-dbus\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.374176 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r59zd\" (UniqueName: \"kubernetes.io/projected/01b00b26-83e4-49bd-bc69-04501dadedd4-kube-api-access-r59zd\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.376280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.375237 2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 09:06:37.406448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.406381 2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gl5lg" Sep 19 09:06:37.413772 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.413742 2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gl5lg" Sep 19 09:06:37.435314 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.435252 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" event={"ID":"5d60026aede2bdb637006ae8e684c1f0","Type":"ContainerStarted","Data":"0c1000653eb3e5bb28ef3a4e4301b0ce14048e06283352885427cebcb7342222"} Sep 19 09:06:37.437060 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.437023 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" event={"ID":"03b66c5394879f7702479f341fee142c","Type":"ContainerStarted","Data":"045acf569aab038bfad597f54b085be432ad965dbb0f8327b181d3ddf39067bf"} Sep 19 09:06:37.461157 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.461123 2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:06:37.474605 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvtwf\" (UniqueName: \"kubernetes.io/projected/71b11675-036f-4c23-a38e-f617f769ea69-kube-api-access-bvtwf\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.474605 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474598 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-kubernetes\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.474870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474622 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-systemd\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.474870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-var-lib-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.474870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474729 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-kubernetes\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.474870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474801 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-systemd\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.474870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474844 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-var-lib-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.474870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474872 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-ovnkube-config\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474911 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fe574be-a073-4960-803c-3d65227fc764-ovn-node-metrics-cert\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474945 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5hvl\" (UniqueName: \"kubernetes.io/projected/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-kube-api-access-x5hvl\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474970 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-device-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.474994 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysctl-d\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475044 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-run-netns\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475098 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8jfg\" (UniqueName: \"kubernetes.io/projected/c5c5af80-f492-4408-aa6f-310bf3fac70d-kube-api-access-m8jfg\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.475155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475144 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475171 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-system-cni-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475205 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjs76\" (UniqueName: \"kubernetes.io/projected/76ea5338-fc9a-45a1-9111-a727e59130fa-kube-api-access-kjs76\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475267 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-conf-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475293 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-lib-modules\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475314 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-tmp\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475350 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-kubelet\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475381 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-systemd\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475405 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkcz\" (UniqueName: \"kubernetes.io/projected/4fe574be-a073-4960-803c-3d65227fc764-kube-api-access-wxkcz\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475431 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475459 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-ovnkube-config\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.475540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475491 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-device-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475572 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-conf-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475693 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475807 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysctl-d\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475852 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-run-netns\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475883 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-run\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475911 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-log-socket\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475938 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475967 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6661651c-f4e1-4a3b-9b68-310a36c28f6b-konnectivity-ca\") pod \"konnectivity-agent-lrw5b\" (UID: \"6661651c-f4e1-4a3b-9b68-310a36c28f6b\") " pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.475995 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476006 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1e4bdae-16ac-478a-be9d-b108434edc8b-host\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476048 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-systemd-units\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.476130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476110 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c5c5af80-f492-4408-aa6f-310bf3fac70d-iptables-alerter-script\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476139 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-k8s-cni-cncf-io\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476168 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-etc-kubernetes\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476174 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-lib-modules\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476192 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysctl-conf\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9szdh\" (UniqueName: \"kubernetes.io/projected/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-kube-api-access-9szdh\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476244 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlnk8\" (UniqueName: \"kubernetes.io/projected/2f96adac-d4cf-4684-b8cf-2a87800cbf10-kube-api-access-tlnk8\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476271 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-system-cni-dir\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476296 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476322 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkv6f\" (UniqueName: \"kubernetes.io/projected/c1e4bdae-16ac-478a-be9d-b108434edc8b-kube-api-access-gkv6f\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476347 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-cni-bin\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476378 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-cni-multus\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476387 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-kubelet\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476404 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f96adac-d4cf-4684-b8cf-2a87800cbf10-hosts-file\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476425 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-k8s-cni-cncf-io\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476430 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f96adac-d4cf-4684-b8cf-2a87800cbf10-tmp-dir\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-kubelet-config\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476470 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-run\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.476716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476500 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-cni-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476535 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-log-socket\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476543 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-multus-certs\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476566 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476578 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476549 2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476607 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-registration-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476633 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6661651c-f4e1-4a3b-9b68-310a36c28f6b-agent-certs\") pod \"konnectivity-agent-lrw5b\" (UID: \"6661651c-f4e1-4a3b-9b68-310a36c28f6b\") " pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476691 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01b00b26-83e4-49bd-bc69-04501dadedd4-cni-binary-copy\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476722 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysconfig\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476749 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-var-lib-kubelet\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476788 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-etc-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476819 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-cni-netd\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476845 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-hostroot\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476018 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-cni-binary-copy\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-host\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476933 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-host\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476078 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-system-cni-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.477564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476949 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476975 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-etc-kubernetes\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.476990 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-node-log\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477001 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-systemd-units\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477016 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-run-ovn-kubernetes\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477019 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6661651c-f4e1-4a3b-9b68-310a36c28f6b-konnectivity-ca\") pod \"konnectivity-agent-lrw5b\" (UID: \"6661651c-f4e1-4a3b-9b68-310a36c28f6b\") " pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-env-overrides\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477051 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1e4bdae-16ac-478a-be9d-b108434edc8b-host\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477073 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-kubelet\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477091 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-cni-bin\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477106 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysctl-conf\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477114 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477145 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-socket-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477161 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-modprobe-d\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477176 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-tuned\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477197 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-slash\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477215 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-ovn\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.478381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477229 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-cnibin\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477245 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-dbus\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477269 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r59zd\" (UniqueName: \"kubernetes.io/projected/01b00b26-83e4-49bd-bc69-04501dadedd4-kube-api-access-r59zd\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477293 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-ovnkube-script-lib\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477319 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-daemon-config\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477362 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-system-cni-dir\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477389 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-env-overrides\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477398 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477406 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-socket-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477433 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-sys-fs\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477477 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-kubelet\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477481 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477506 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-node-log\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477559 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-run-ovn-kubernetes\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477566 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-cni-bin\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477595 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-sys\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477625 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5c5af80-f492-4408-aa6f-310bf3fac70d-host-slash\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477650 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1e4bdae-16ac-478a-be9d-b108434edc8b-serviceca\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.479158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477673 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-cnibin\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477697 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-socket-dir-parent\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477722 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-netns\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477780 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-daemon-config\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477823 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-netns\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477836 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-sys-fs\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477881 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-sys\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477899 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-modprobe-d\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.477918 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5c5af80-f492-4408-aa6f-310bf3fac70d-host-slash\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478046 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c5c5af80-f492-4408-aa6f-310bf3fac70d-iptables-alerter-script\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478137 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/76ea5338-fc9a-45a1-9111-a727e59130fa-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478198 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478207 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-cnibin\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478256 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-socket-dir-parent\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478306 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478341 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-cni-bin\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.476346 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478379 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1e4bdae-16ac-478a-be9d-b108434edc8b-serviceca\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.479990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478449 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-var-lib-kubelet\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.478466 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret podName:6f74b8f0-71dd-482c-aa43-b65acd2cb16b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:37.978442771 +0000 UTC m=+4.736563802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret") pod "global-pull-secret-syncer-fljvj" (UID: "6f74b8f0-71dd-482c-aa43-b65acd2cb16b") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478486 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-etc-openvswitch\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478499 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-systemd\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478532 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-cni-netd\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478557 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-var-lib-cni-multus\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478580 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-hostroot\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478600 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f96adac-d4cf-4684-b8cf-2a87800cbf10-hosts-file\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478638 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-host-run-multus-certs\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.477487 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478664 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/01b00b26-83e4-49bd-bc69-04501dadedd4-multus-cni-dir\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.478686 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs podName:3f5f3742-e891-4188-aa1d-c48c5e8e8a0b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:37.978673802 +0000 UTC m=+4.736794818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs") pod "network-metrics-daemon-nst5s" (UID: "3f5f3742-e891-4188-aa1d-c48c5e8e8a0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478834 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-kubelet-config\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478862 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76ea5338-fc9a-45a1-9111-a727e59130fa-cnibin\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478903 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-host-slash\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478905 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f96adac-d4cf-4684-b8cf-2a87800cbf10-tmp-dir\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.478969 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4fe574be-a073-4960-803c-3d65227fc764-run-ovn\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.480826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.479023 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/71b11675-036f-4c23-a38e-f617f769ea69-registration-dir\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.481636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.479335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-sysconfig\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.481636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.479490 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/01b00b26-83e4-49bd-bc69-04501dadedd4-cni-binary-copy\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.481636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.479511 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4fe574be-a073-4960-803c-3d65227fc764-ovnkube-script-lib\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.481636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.479591 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-dbus\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.481636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.481512 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fe574be-a073-4960-803c-3d65227fc764-ovn-node-metrics-cert\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.481636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.481614 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-etc-tuned\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.481997 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.481966 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6661651c-f4e1-4a3b-9b68-310a36c28f6b-agent-certs\") pod \"konnectivity-agent-lrw5b\" (UID: \"6661651c-f4e1-4a3b-9b68-310a36c28f6b\") " pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.482558 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.482536 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-tmp\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.488612 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.488538 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvtwf\" (UniqueName: \"kubernetes.io/projected/71b11675-036f-4c23-a38e-f617f769ea69-kube-api-access-bvtwf\") pod \"aws-ebs-csi-driver-node-bgkc9\" (UID: \"71b11675-036f-4c23-a38e-f617f769ea69\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.488790 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.488760 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9szdh\" (UniqueName: \"kubernetes.io/projected/ce3904e4-2ee3-4ab6-bdee-d503eb9018fa-kube-api-access-9szdh\") pod \"tuned-9nfpt\" (UID: \"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa\") " pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.489228 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.489179 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjs76\" (UniqueName: \"kubernetes.io/projected/76ea5338-fc9a-45a1-9111-a727e59130fa-kube-api-access-kjs76\") pod \"multus-additional-cni-plugins-chtr4\" (UID: \"76ea5338-fc9a-45a1-9111-a727e59130fa\") " pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.490128 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.490096 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8jfg\" (UniqueName: \"kubernetes.io/projected/c5c5af80-f492-4408-aa6f-310bf3fac70d-kube-api-access-m8jfg\") pod \"iptables-alerter-tqpch\" (UID: \"c5c5af80-f492-4408-aa6f-310bf3fac70d\") " pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.492180 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.491956 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:06:37.492180 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.491991 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:06:37.492180 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.492011 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2wzv4 for pod openshift-network-diagnostics/network-check-target-gml7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:37.492180 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.492133 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4 podName:fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:37.992108585 +0000 UTC m=+4.750229614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2wzv4" (UniqueName: "kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4") pod "network-check-target-gml7f" (UID: "fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:37.494626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.494392 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlnk8\" (UniqueName: \"kubernetes.io/projected/2f96adac-d4cf-4684-b8cf-2a87800cbf10-kube-api-access-tlnk8\") pod \"node-resolver-wcqst\" (UID: \"2f96adac-d4cf-4684-b8cf-2a87800cbf10\") " pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.494853 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.494805 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5hvl\" (UniqueName: \"kubernetes.io/projected/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-kube-api-access-x5hvl\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:37.496152 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.496051 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkv6f\" (UniqueName: \"kubernetes.io/projected/c1e4bdae-16ac-478a-be9d-b108434edc8b-kube-api-access-gkv6f\") pod \"node-ca-kghq6\" (UID: \"c1e4bdae-16ac-478a-be9d-b108434edc8b\") " pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.496514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.496350 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxkcz\" (UniqueName: \"kubernetes.io/projected/4fe574be-a073-4960-803c-3d65227fc764-kube-api-access-wxkcz\") pod \"ovnkube-node-6c442\" (UID: \"4fe574be-a073-4960-803c-3d65227fc764\") " pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.497828 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.497810 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r59zd\" (UniqueName: \"kubernetes.io/projected/01b00b26-83e4-49bd-bc69-04501dadedd4-kube-api-access-r59zd\") pod \"multus-5jch7\" (UID: \"01b00b26-83e4-49bd-bc69-04501dadedd4\") " pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.648019 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.647911 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" Sep 19 09:06:37.658010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.657970 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wcqst" Sep 19 09:06:37.668065 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.668024 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kghq6" Sep 19 09:06:37.676028 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.675996 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5jch7" Sep 19 09:06:37.682974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.682948 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:06:37.697015 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.696980 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tqpch" Sep 19 09:06:37.706242 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.706208 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-chtr4" Sep 19 09:06:37.715198 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.715163 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:37.722287 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.722251 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" Sep 19 09:06:37.982169 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.982078 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:37.982169 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:37.982132 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:37.982349 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.982262 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:37.982349 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.982265 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:37.982349 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.982323 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret podName:6f74b8f0-71dd-482c-aa43-b65acd2cb16b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:38.982307228 +0000 UTC m=+5.740428244 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret") pod "global-pull-secret-syncer-fljvj" (UID: "6f74b8f0-71dd-482c-aa43-b65acd2cb16b") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:37.982349 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:37.982338 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs podName:3f5f3742-e891-4188-aa1d-c48c5e8e8a0b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:38.982331 +0000 UTC m=+5.740452003 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs") pod "network-metrics-daemon-nst5s" (UID: "3f5f3742-e891-4188-aa1d-c48c5e8e8a0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:38.083252 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:38.083208 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:38.083463 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.083387 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:06:38.083463 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.083413 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:06:38.083463 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.083427 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2wzv4 for pod openshift-network-diagnostics/network-check-target-gml7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:38.083644 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.083491 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4 podName:fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:39.083473134 +0000 UTC m=+5.841594148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2wzv4" (UniqueName: "kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4") pod "network-check-target-gml7f" (UID: "fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:38.417398 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:38.417350 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 09:01:37 +0000 UTC" deadline="2028-06-18 22:46:17.082453762 +0000 UTC" Sep 19 09:06:38.417398 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:38.417386 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15325h39m38.665071976s" Sep 19 09:06:38.429648 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:38.429614 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:38.429781 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.429760 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:38.482155 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:38.482109 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f96adac_d4cf_4684_b8cf_2a87800cbf10.slice/crio-a5221dad34384a5de345b80c862dabad253f3fd2d5fbe91c669f9e6c73d9fb0d WatchSource:0}: Error finding container a5221dad34384a5de345b80c862dabad253f3fd2d5fbe91c669f9e6c73d9fb0d: Status 404 returned error can't find the container with id a5221dad34384a5de345b80c862dabad253f3fd2d5fbe91c669f9e6c73d9fb0d Sep 19 09:06:38.484452 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:38.484408 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b00b26_83e4_49bd_bc69_04501dadedd4.slice/crio-3f0e3621bbb0c9f1ad8bf2c4c050037cf058966e1c4ecb2e8c4faeea9fad8055 WatchSource:0}: Error finding container 3f0e3621bbb0c9f1ad8bf2c4c050037cf058966e1c4ecb2e8c4faeea9fad8055: Status 404 returned error can't find the container with id 3f0e3621bbb0c9f1ad8bf2c4c050037cf058966e1c4ecb2e8c4faeea9fad8055 Sep 19 09:06:38.489578 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:38.489547 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6661651c_f4e1_4a3b_9b68_310a36c28f6b.slice/crio-26b15ce51c7e499dfd7dd3c94b4047670ceb8c6506253ec253b58eb45aff7c94 WatchSource:0}: Error finding container 26b15ce51c7e499dfd7dd3c94b4047670ceb8c6506253ec253b58eb45aff7c94: Status 404 returned error can't find the container with id 26b15ce51c7e499dfd7dd3c94b4047670ceb8c6506253ec253b58eb45aff7c94 Sep 19 09:06:38.490962 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:38.490933 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ea5338_fc9a_45a1_9111_a727e59130fa.slice/crio-77135467024d5a392158948e98ee1b45075e4decd8a03458b36b8f06c879b1f7 WatchSource:0}: Error finding container 77135467024d5a392158948e98ee1b45075e4decd8a03458b36b8f06c879b1f7: Status 404 returned error can't find the container with id 77135467024d5a392158948e98ee1b45075e4decd8a03458b36b8f06c879b1f7 Sep 19 09:06:38.492787 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:38.492492 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c5af80_f492_4408_aa6f_310bf3fac70d.slice/crio-d762a0468058c20527c25a6f60a5986da24239959c91fb58bd6d360374d1b490 WatchSource:0}: Error finding container d762a0468058c20527c25a6f60a5986da24239959c91fb58bd6d360374d1b490: Status 404 returned error can't find the container with id d762a0468058c20527c25a6f60a5986da24239959c91fb58bd6d360374d1b490 Sep 19 09:06:38.493253 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:38.493168 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe574be_a073_4960_803c_3d65227fc764.slice/crio-22b53e18b62e2d7eba3d17dc2edc333dbde5fee0dabddeacc506aa6b35d7c131 WatchSource:0}: Error finding container 22b53e18b62e2d7eba3d17dc2edc333dbde5fee0dabddeacc506aa6b35d7c131: Status 404 returned error can't find the container with id 22b53e18b62e2d7eba3d17dc2edc333dbde5fee0dabddeacc506aa6b35d7c131 Sep 19 09:06:38.495094 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:06:38.495020 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b11675_036f_4c23_a38e_f617f769ea69.slice/crio-9ee108bc464cdaeaa24f3044de5e9a31e2b49909dc3c4562498cd61d005daf76 WatchSource:0}: Error finding container 9ee108bc464cdaeaa24f3044de5e9a31e2b49909dc3c4562498cd61d005daf76: Status 404 returned error can't find the container with id 9ee108bc464cdaeaa24f3044de5e9a31e2b49909dc3c4562498cd61d005daf76 Sep 19 09:06:38.990740 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:38.990404 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:38.990911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:38.990741 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:38.990911 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.990882 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:38.991033 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.990944 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret podName:6f74b8f0-71dd-482c-aa43-b65acd2cb16b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:40.990927153 +0000 UTC m=+7.749048161 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret") pod "global-pull-secret-syncer-fljvj" (UID: "6f74b8f0-71dd-482c-aa43-b65acd2cb16b") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:38.991358 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.991341 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:38.991482 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:38.991394 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs podName:3f5f3742-e891-4188-aa1d-c48c5e8e8a0b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:40.991381484 +0000 UTC m=+7.749502500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs") pod "network-metrics-daemon-nst5s" (UID: "3f5f3742-e891-4188-aa1d-c48c5e8e8a0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:39.092122 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.092052 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:39.092369 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:39.092342 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:06:39.092438 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:39.092373 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:06:39.092438 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:39.092387 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2wzv4 for pod openshift-network-diagnostics/network-check-target-gml7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:39.092558 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:39.092475 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4 podName:fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:41.09245154 +0000 UTC m=+7.850572559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2wzv4" (UniqueName: "kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4") pod "network-check-target-gml7f" (UID: "fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:39.417855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.417778 2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 09:01:37 +0000 UTC" deadline="2028-03-25 00:58:02.71145377 +0000 UTC" Sep 19 09:06:39.417855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.417823 2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13263h51m23.293635232s" Sep 19 09:06:39.432537 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.432495 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:39.432732 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:39.432645 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:39.433070 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.433049 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:39.433189 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:39.433159 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:39.437581 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.437359 2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:06:39.465086 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.465045 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lrw5b" event={"ID":"6661651c-f4e1-4a3b-9b68-310a36c28f6b","Type":"ContainerStarted","Data":"26b15ce51c7e499dfd7dd3c94b4047670ceb8c6506253ec253b58eb45aff7c94"} Sep 19 09:06:39.481297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.481184 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" event={"ID":"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa","Type":"ContainerStarted","Data":"f0b770f36c7140300ae8a16dbb109104b4eec500ca749eb2119c431d8c41fe81"} Sep 19 09:06:39.510019 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.509971 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5jch7" event={"ID":"01b00b26-83e4-49bd-bc69-04501dadedd4","Type":"ContainerStarted","Data":"3f0e3621bbb0c9f1ad8bf2c4c050037cf058966e1c4ecb2e8c4faeea9fad8055"} Sep 19 09:06:39.526085 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.526039 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kghq6" event={"ID":"c1e4bdae-16ac-478a-be9d-b108434edc8b","Type":"ContainerStarted","Data":"5db0214e85495738d8da2cebda86be6a9c23a36897816f032efa5ee2a9444460"} Sep 19 09:06:39.540538 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.540483 2585 generic.go:356] "Generic (PLEG): container finished" podID="5d60026aede2bdb637006ae8e684c1f0" containerID="74018e8cc73ad2b759fbb14775abd349a87e9afc53d7489ebfe9fc297a478fae" exitCode=0 Sep 19 09:06:39.541586 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.541402 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" event={"ID":"5d60026aede2bdb637006ae8e684c1f0","Type":"ContainerDied","Data":"74018e8cc73ad2b759fbb14775abd349a87e9afc53d7489ebfe9fc297a478fae"} Sep 19 09:06:39.545887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.545855 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"22b53e18b62e2d7eba3d17dc2edc333dbde5fee0dabddeacc506aa6b35d7c131"} Sep 19 09:06:39.548224 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.548125 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tqpch" event={"ID":"c5c5af80-f492-4408-aa6f-310bf3fac70d","Type":"ContainerStarted","Data":"d762a0468058c20527c25a6f60a5986da24239959c91fb58bd6d360374d1b490"} Sep 19 09:06:39.554261 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.554158 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wcqst" event={"ID":"2f96adac-d4cf-4684-b8cf-2a87800cbf10","Type":"ContainerStarted","Data":"a5221dad34384a5de345b80c862dabad253f3fd2d5fbe91c669f9e6c73d9fb0d"} Sep 19 09:06:39.570777 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.570734 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" event={"ID":"03b66c5394879f7702479f341fee142c","Type":"ContainerStarted","Data":"30fc5e8c500f2e956830dd036651701eacd941759d882c477b7c14238e845fa3"} Sep 19 09:06:39.592313 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.592267 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" event={"ID":"71b11675-036f-4c23-a38e-f617f769ea69","Type":"ContainerStarted","Data":"9ee108bc464cdaeaa24f3044de5e9a31e2b49909dc3c4562498cd61d005daf76"} Sep 19 09:06:39.597067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:39.597018 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerStarted","Data":"77135467024d5a392158948e98ee1b45075e4decd8a03458b36b8f06c879b1f7"} Sep 19 09:06:40.430016 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:40.429977 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:40.430502 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:40.430110 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:40.609292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:40.609215 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" event={"ID":"5d60026aede2bdb637006ae8e684c1f0","Type":"ContainerStarted","Data":"21d82178149cc8bd9a1f421a939d869f60384cb5718b359e825daff7b86d8b9b"} Sep 19 09:06:40.629802 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:40.629737 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-57.ec2.internal" podStartSLOduration=4.629716106 podStartE2EDuration="4.629716106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:40.628763162 +0000 UTC m=+7.386884190" watchObservedRunningTime="2026-09-19 09:06:40.629716106 +0000 UTC m=+7.387837131" Sep 19 09:06:40.630001 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:40.629947 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-57.ec2.internal" podStartSLOduration=4.629935313 podStartE2EDuration="4.629935313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:39.589095774 +0000 UTC m=+6.347216803" watchObservedRunningTime="2026-09-19 09:06:40.629935313 +0000 UTC m=+7.388056340" Sep 19 09:06:41.008246 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:41.008205 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:41.008445 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:41.008276 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:41.008445 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.008422 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:41.008581 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.008488 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret podName:6f74b8f0-71dd-482c-aa43-b65acd2cb16b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:45.008467498 +0000 UTC m=+11.766588523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret") pod "global-pull-secret-syncer-fljvj" (UID: "6f74b8f0-71dd-482c-aa43-b65acd2cb16b") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:41.009036 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.008957 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:41.009155 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.009132 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs podName:3f5f3742-e891-4188-aa1d-c48c5e8e8a0b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:45.009101074 +0000 UTC m=+11.767222077 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs") pod "network-metrics-daemon-nst5s" (UID: "3f5f3742-e891-4188-aa1d-c48c5e8e8a0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:41.109336 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:41.108691 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:41.109336 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.108865 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:06:41.109336 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.108883 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:06:41.109336 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.108895 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2wzv4 for pod openshift-network-diagnostics/network-check-target-gml7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:41.109336 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.108967 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4 podName:fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:45.108947769 +0000 UTC m=+11.867068778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2wzv4" (UniqueName: "kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4") pod "network-check-target-gml7f" (UID: "fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:41.431403 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:41.430675 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:41.431403 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:41.430719 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:41.431403 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.430826 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:41.431403 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:41.431321 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:42.430314 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:42.430203 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:42.430505 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:42.430343 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:43.430337 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:43.430287 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:43.430837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:43.430300 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:43.431545 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:43.431169 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:43.431545 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:43.431492 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:44.429980 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:44.429937 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:44.430182 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:44.430098 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:45.046786 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:45.046752 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:45.047265 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:45.046803 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:45.047265 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.046915 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:45.047265 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.046918 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:45.047265 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.046990 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret podName:6f74b8f0-71dd-482c-aa43-b65acd2cb16b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:53.04697168 +0000 UTC m=+19.805092700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret") pod "global-pull-secret-syncer-fljvj" (UID: "6f74b8f0-71dd-482c-aa43-b65acd2cb16b") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:45.047265 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.047011 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs podName:3f5f3742-e891-4188-aa1d-c48c5e8e8a0b nodeName:}" failed. No retries permitted until 2026-09-19 09:06:53.047001676 +0000 UTC m=+19.805122693 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs") pod "network-metrics-daemon-nst5s" (UID: "3f5f3742-e891-4188-aa1d-c48c5e8e8a0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:45.147951 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:45.147909 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:45.148145 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.148059 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:06:45.148145 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.148080 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:06:45.148145 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.148089 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2wzv4 for pod openshift-network-diagnostics/network-check-target-gml7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:45.148145 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.148139 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4 podName:fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:53.148119501 +0000 UTC m=+19.906240526 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2wzv4" (UniqueName: "kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4") pod "network-check-target-gml7f" (UID: "fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:45.431535 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:45.431448 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:45.431735 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.431594 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:45.432000 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:45.431967 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:45.432102 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:45.432084 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:46.430456 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:46.430410 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:46.430860 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:46.430569 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:47.431322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:47.430682 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:47.431322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:47.430749 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:47.431322 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:47.430884 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:47.431322 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:47.430938 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:48.430214 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:48.430178 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:48.430422 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:48.430292 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:49.429823 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:49.429784 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:49.430301 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:49.429838 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:49.430301 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:49.429935 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:49.430301 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:49.430056 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:50.429568 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:50.429529 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:50.429758 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:50.429660 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:51.429839 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:51.429799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:51.430268 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:51.429847 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:51.430268 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:51.429953 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:51.430268 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:51.430057 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:52.430497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:52.430448 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:52.430907 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:52.430583 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:53.111770 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:53.111727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:53.111975 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:53.111782 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:53.111975 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.111902 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:53.112092 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.111980 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs podName:3f5f3742-e891-4188-aa1d-c48c5e8e8a0b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:09.111962824 +0000 UTC m=+35.870083827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs") pod "network-metrics-daemon-nst5s" (UID: "3f5f3742-e891-4188-aa1d-c48c5e8e8a0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:06:53.112092 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.111914 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:53.112092 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.112065 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret podName:6f74b8f0-71dd-482c-aa43-b65acd2cb16b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:09.112038198 +0000 UTC m=+35.870159208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret") pod "global-pull-secret-syncer-fljvj" (UID: "6f74b8f0-71dd-482c-aa43-b65acd2cb16b") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:06:53.213000 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:53.212963 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:53.213199 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.213147 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:06:53.213199 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.213169 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:06:53.213199 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.213181 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2wzv4 for pod openshift-network-diagnostics/network-check-target-gml7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:53.213307 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.213232 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4 podName:fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:09.213218594 +0000 UTC m=+35.971339598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2wzv4" (UniqueName: "kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4") pod "network-check-target-gml7f" (UID: "fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:06:53.430415 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:53.430325 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:53.430415 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:53.430370 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:53.430967 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.430473 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:53.430967 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:53.430627 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:54.430015 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:54.429980 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:54.430185 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:54.430092 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:55.430319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.430289 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:55.430687 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:55.430375 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:55.430744 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.430694 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:55.430840 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:55.430802 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:55.644009 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.643972 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kghq6" event={"ID":"c1e4bdae-16ac-478a-be9d-b108434edc8b","Type":"ContainerStarted","Data":"ce5cc11d39450fb61348df1316888652c5107c9559ee3c6c0fc68338a0144189"} Sep 19 09:06:55.645350 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.645320 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wcqst" event={"ID":"2f96adac-d4cf-4684-b8cf-2a87800cbf10","Type":"ContainerStarted","Data":"fd863896e8569c38a51aa7fe09d89139673345c397b921a183231d36f949ebf7"} Sep 19 09:06:55.646826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.646804 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" event={"ID":"71b11675-036f-4c23-a38e-f617f769ea69","Type":"ContainerStarted","Data":"22d6e416a90a095b6fa9696d54e5ea3a85c3597e9aea847c1083b33057f180b4"} Sep 19 09:06:55.648281 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.648253 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerStarted","Data":"29d02268b26c22655fc80d5c32e1cd1e520d7af8e19ae8be24c5cd1174bcad80"} Sep 19 09:06:55.649638 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.649607 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lrw5b" event={"ID":"6661651c-f4e1-4a3b-9b68-310a36c28f6b","Type":"ContainerStarted","Data":"5723b78c18a15421708264a807de9ef1e2f4a55b00212d23c4e8d0ed74f4598d"} Sep 19 09:06:55.670269 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.670031 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kghq6" podStartSLOduration=3.834688297 podStartE2EDuration="20.670011718s" totalImagesPullingTime="16.835323421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:55.669953863 +0000 UTC m=+22.428074889" watchObservedRunningTime="2026-09-19 09:06:55.670011718 +0000 UTC m=+22.428132743" Sep 19 09:06:55.696300 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.696151 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wcqst" podStartSLOduration=3.904925087 podStartE2EDuration="20.696129677s" totalImagesPullingTime="16.79120459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:55.695318662 +0000 UTC m=+22.453439678" watchObservedRunningTime="2026-09-19 09:06:55.696129677 +0000 UTC m=+22.454250734" Sep 19 09:06:55.970371 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.970348 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:55.970871 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:55.970852 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:56.001695 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.001640 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lrw5b" podStartSLOduration=4.217274592 podStartE2EDuration="21.001615628s" totalImagesPullingTime="16.784341036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:55.760296864 +0000 UTC m=+22.518417889" watchObservedRunningTime="2026-09-19 09:06:56.001615628 +0000 UTC m=+22.759736654" Sep 19 09:06:56.430053 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.429847 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:56.430196 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:56.430129 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:56.627574 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.627553 2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 19 09:06:56.654180 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654160 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:06:56.654503 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654481 2585 generic.go:356] "Generic (PLEG): container finished" podID="4fe574be-a073-4960-803c-3d65227fc764" containerID="77c82ec08ffa5266fb27f98eff5094e071a077eaaa9ad8b89d02d4915b02f84e" exitCode=1 Sep 19 09:06:56.654594 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654552 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"ce29b9c1c01672bf775457bf334452c6da3701bab1fc58b8e027e1f0438ce5ee"} Sep 19 09:06:56.654652 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654614 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"a80bc4f659fb2edf7909fb5e45fc1b34463c38433eccd10c8025bd88b6e3f14a"} Sep 19 09:06:56.654693 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654651 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"04be8728b16cb94903dbbcbaa3361f9d6bfc2e0deed828f66bd80b4c4b6b1479"} Sep 19 09:06:56.654693 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654686 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"b0a246f98c41cc7418bc26f08834441d27b6d9f71ae152d7571994a4bcc383e0"} Sep 19 09:06:56.654755 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654724 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerDied","Data":"77c82ec08ffa5266fb27f98eff5094e071a077eaaa9ad8b89d02d4915b02f84e"} Sep 19 09:06:56.654795 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.654755 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"d1cd60f5f1d18588df64a158399f7920f91b81829f5a4b4bc227544eda0587c8"} Sep 19 09:06:56.656030 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.656003 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tqpch" event={"ID":"c5c5af80-f492-4408-aa6f-310bf3fac70d","Type":"ContainerStarted","Data":"bc078381f0f8542ddddd23d73eba0d1dac58061298494db99d8d2336a2cbc303"} Sep 19 09:06:56.657798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.657776 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" event={"ID":"71b11675-036f-4c23-a38e-f617f769ea69","Type":"ContainerStarted","Data":"4499abe27e621a711262bccbb904195c1e07740d469dbdb4633af2dd4a47d6ab"} Sep 19 09:06:56.659174 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.659156 2585 generic.go:356] "Generic (PLEG): container finished" podID="76ea5338-fc9a-45a1-9111-a727e59130fa" containerID="29d02268b26c22655fc80d5c32e1cd1e520d7af8e19ae8be24c5cd1174bcad80" exitCode=0 Sep 19 09:06:56.659260 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.659220 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerDied","Data":"29d02268b26c22655fc80d5c32e1cd1e520d7af8e19ae8be24c5cd1174bcad80"} Sep 19 09:06:56.660577 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.660538 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" event={"ID":"ce3904e4-2ee3-4ab6-bdee-d503eb9018fa","Type":"ContainerStarted","Data":"fe5507ea8925955b2cd99b004a3b464a0c52dbc461449d99a2004a2722476dcc"} Sep 19 09:06:56.662013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.661991 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5jch7" event={"ID":"01b00b26-83e4-49bd-bc69-04501dadedd4","Type":"ContainerStarted","Data":"9b5eeb61bb393e15a284db37071eec0f715eb004ae9186f17ec1f3e5c3173708"} Sep 19 09:06:56.662485 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.662471 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:56.662574 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.662530 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lrw5b" Sep 19 09:06:56.675448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.675410 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tqpch" podStartSLOduration=4.845976883 podStartE2EDuration="21.675396988s" totalImagesPullingTime="16.829420105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:56.675098127 +0000 UTC m=+23.433219150" watchObservedRunningTime="2026-09-19 09:06:56.675396988 +0000 UTC m=+23.433518014" Sep 19 09:06:56.700015 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.699976 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5jch7" podStartSLOduration=4.48339934 podStartE2EDuration="21.699965187s" totalImagesPullingTime="17.216565847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:56.699786181 +0000 UTC m=+23.457907205" watchObservedRunningTime="2026-09-19 09:06:56.699965187 +0000 UTC m=+23.458086211" Sep 19 09:06:56.775148 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:56.775101 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9nfpt" podStartSLOduration=4.937789098 podStartE2EDuration="21.775088907s" totalImagesPullingTime="16.837299809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:56.774826703 +0000 UTC m=+23.532947729" watchObservedRunningTime="2026-09-19 09:06:56.775088907 +0000 UTC m=+23.533209931" Sep 19 09:06:57.430224 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:57.430187 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:57.430401 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:57.430312 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:06:57.430401 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:57.430336 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:57.430510 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:57.430442 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:57.459089 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:57.458911 2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T09:06:56.627570343Z","UUID":"b263efcc-950b-4718-939d-9bac798bae98","Handler":null,"Name":"","Endpoint":""} Sep 19 09:06:57.460712 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:57.460675 2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 19 09:06:57.460842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:57.460722 2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 19 09:06:58.430093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:58.430062 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:06:58.430477 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:58.430184 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:06:58.668393 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:58.668307 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" event={"ID":"71b11675-036f-4c23-a38e-f617f769ea69","Type":"ContainerStarted","Data":"fea9bee27b7d6a8061bac80bb709b600e881653d6f719a49f3343da8608530d4"} Sep 19 09:06:58.672348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:58.672172 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:06:58.672818 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:58.672789 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"3672d48e3ed1d24188db615e494d1b08499259b16402aca1843a93202756bc66"} Sep 19 09:06:58.691652 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:58.691617 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgkc9" podStartSLOduration=4.971565151 podStartE2EDuration="23.691603319s" totalImagesPullingTime="18.720038168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:58.691125954 +0000 UTC m=+25.449246979" watchObservedRunningTime="2026-09-19 09:06:58.691603319 +0000 UTC m=+25.449724367" Sep 19 09:06:59.429746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:59.429715 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:06:59.429925 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:06:59.429720 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:06:59.429925 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:59.429851 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:06:59.429925 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:06:59.429911 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:07:00.430171 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:00.430143 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:00.430687 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:00.430257 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:07:01.430366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.430286 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:01.430997 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.430295 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:01.430997 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:01.430399 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:07:01.430997 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:01.430489 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:07:01.681892 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.681568 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerStarted","Data":"947e7a71150fe59406a0fcd2cd22bfbd7fffd2b283775e30859af2e39399a5b4"} Sep 19 09:07:01.685181 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.685161 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:07:01.685545 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.685510 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"a415eb0d03ef5862d28a9fd592954b4fdbf52451a00259c3f375317f5436ea14"} Sep 19 09:07:01.685769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.685753 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:07:01.685825 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.685805 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:07:01.685937 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.685914 2585 scope.go:117] "RemoveContainer" containerID="77c82ec08ffa5266fb27f98eff5094e071a077eaaa9ad8b89d02d4915b02f84e" Sep 19 09:07:01.701834 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:01.701815 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:07:02.429918 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.429831 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:02.430078 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:02.429949 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:07:02.692652 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.692583 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:07:02.693129 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.693098 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" event={"ID":"4fe574be-a073-4960-803c-3d65227fc764","Type":"ContainerStarted","Data":"11e7670c0d53895e45fe3267ca2badb813a8bb4b73505c3931548eca4f8d468d"} Sep 19 09:07:02.693348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.693330 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:07:02.695250 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.695226 2585 generic.go:356] "Generic (PLEG): container finished" podID="76ea5338-fc9a-45a1-9111-a727e59130fa" containerID="947e7a71150fe59406a0fcd2cd22bfbd7fffd2b283775e30859af2e39399a5b4" exitCode=0 Sep 19 09:07:02.695341 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.695272 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerDied","Data":"947e7a71150fe59406a0fcd2cd22bfbd7fffd2b283775e30859af2e39399a5b4"} Sep 19 09:07:02.707856 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.707837 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:07:02.747304 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:02.747262 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" podStartSLOduration=10.878327336 podStartE2EDuration="27.747247268s" totalImagesPullingTime="16.868919932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:02.746924706 +0000 UTC m=+29.505045730" watchObservedRunningTime="2026-09-19 09:07:02.747247268 +0000 UTC m=+29.505368292" Sep 19 09:07:03.246319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:03.246288 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gml7f"] Sep 19 09:07:03.246507 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:03.246419 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:03.246507 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:03.246497 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:07:03.251732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:03.251700 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fljvj"] Sep 19 09:07:03.251863 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:03.251819 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:03.251941 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:03.251919 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:07:03.272253 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:03.272183 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nst5s"] Sep 19 09:07:03.272369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:03.272311 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:03.272470 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:03.272419 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:07:04.430482 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:04.430449 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:04.430902 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:04.430567 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:07:04.704824 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:04.704751 2585 generic.go:356] "Generic (PLEG): container finished" podID="76ea5338-fc9a-45a1-9111-a727e59130fa" containerID="435d2f89cb11e9e4a369c83257565328a87ae5e50e2715cd01ae41f0a0e2f3b8" exitCode=0 Sep 19 09:07:04.704965 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:04.704826 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerDied","Data":"435d2f89cb11e9e4a369c83257565328a87ae5e50e2715cd01ae41f0a0e2f3b8"} Sep 19 09:07:05.431213 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:05.431011 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:05.431660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:05.431091 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:05.431788 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:05.431728 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:07:05.433824 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:05.432074 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:07:06.430616 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:06.430577 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:06.430788 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:06.430707 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:07:06.710968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:06.710881 2585 generic.go:356] "Generic (PLEG): container finished" podID="76ea5338-fc9a-45a1-9111-a727e59130fa" containerID="a6955fbd42288d493da57680f9034dd09c1a41173d44e492f17085c7abbdc24d" exitCode=0 Sep 19 09:07:06.710968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:06.710942 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerDied","Data":"a6955fbd42288d493da57680f9034dd09c1a41173d44e492f17085c7abbdc24d"} Sep 19 09:07:07.430125 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:07.430094 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:07.430125 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:07.430120 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:07.430351 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:07.430203 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gml7f" podUID="fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0" Sep 19 09:07:07.430351 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:07.430320 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nst5s" podUID="3f5f3742-e891-4188-aa1d-c48c5e8e8a0b" Sep 19 09:07:08.429847 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.429766 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:08.430397 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:08.429915 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fljvj" podUID="6f74b8f0-71dd-482c-aa43-b65acd2cb16b" Sep 19 09:07:08.549369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.549337 2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-57.ec2.internal" event="NodeReady" Sep 19 09:07:08.549554 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.549492 2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 19 09:07:08.711439 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.711364 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zvr64"] Sep 19 09:07:08.737584 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.737553 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-48xg4"] Sep 19 09:07:08.737755 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.737684 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.746366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.746342 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 19 09:07:08.746498 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.746448 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 19 09:07:08.746588 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.746515 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xchdv\"" Sep 19 09:07:08.757652 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.757629 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvr64"] Sep 19 09:07:08.757800 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.757730 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:08.762987 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.762963 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 19 09:07:08.763111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.763010 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 19 09:07:08.763305 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.763272 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5nz4g\"" Sep 19 09:07:08.763570 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.763501 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 19 09:07:08.805783 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.805754 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-48xg4"] Sep 19 09:07:08.819071 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.819049 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cf31f40-1983-41a0-8bdd-88249a000156-tmp-dir\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.819212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.819078 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4msr8\" (UniqueName: \"kubernetes.io/projected/7cf31f40-1983-41a0-8bdd-88249a000156-kube-api-access-4msr8\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.819212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.819125 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf31f40-1983-41a0-8bdd-88249a000156-config-volume\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.819212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.819179 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.920076 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920042 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf31f40-1983-41a0-8bdd-88249a000156-config-volume\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.920257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920087 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.920257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920122 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gxsx\" (UniqueName: \"kubernetes.io/projected/c40b15ed-37e4-476a-a915-ae88b2ef8e37-kube-api-access-5gxsx\") pod \"ingress-canary-48xg4\" (UID: \"c40b15ed-37e4-476a-a915-ae88b2ef8e37\") " pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:08.920257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920175 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c40b15ed-37e4-476a-a915-ae88b2ef8e37-cert\") pod \"ingress-canary-48xg4\" (UID: \"c40b15ed-37e4-476a-a915-ae88b2ef8e37\") " pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:08.920257 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:08.920230 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:07:08.920257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cf31f40-1983-41a0-8bdd-88249a000156-tmp-dir\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.920505 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:08.920309 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls podName:7cf31f40-1983-41a0-8bdd-88249a000156 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:09.420290895 +0000 UTC m=+36.178411911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls") pod "dns-default-zvr64" (UID: "7cf31f40-1983-41a0-8bdd-88249a000156") : secret "dns-default-metrics-tls" not found Sep 19 09:07:08.920505 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4msr8\" (UniqueName: \"kubernetes.io/projected/7cf31f40-1983-41a0-8bdd-88249a000156-kube-api-access-4msr8\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.920696 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920593 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cf31f40-1983-41a0-8bdd-88249a000156-tmp-dir\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.920904 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.920875 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7cf31f40-1983-41a0-8bdd-88249a000156-config-volume\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:08.936747 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:08.936716 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4msr8\" (UniqueName: \"kubernetes.io/projected/7cf31f40-1983-41a0-8bdd-88249a000156-kube-api-access-4msr8\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:09.021791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.021707 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gxsx\" (UniqueName: \"kubernetes.io/projected/c40b15ed-37e4-476a-a915-ae88b2ef8e37-kube-api-access-5gxsx\") pod \"ingress-canary-48xg4\" (UID: \"c40b15ed-37e4-476a-a915-ae88b2ef8e37\") " pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:09.021791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.021765 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c40b15ed-37e4-476a-a915-ae88b2ef8e37-cert\") pod \"ingress-canary-48xg4\" (UID: \"c40b15ed-37e4-476a-a915-ae88b2ef8e37\") " pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:09.024493 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.024468 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c40b15ed-37e4-476a-a915-ae88b2ef8e37-cert\") pod \"ingress-canary-48xg4\" (UID: \"c40b15ed-37e4-476a-a915-ae88b2ef8e37\") " pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:09.037853 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.037829 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gxsx\" (UniqueName: \"kubernetes.io/projected/c40b15ed-37e4-476a-a915-ae88b2ef8e37-kube-api-access-5gxsx\") pod \"ingress-canary-48xg4\" (UID: \"c40b15ed-37e4-476a-a915-ae88b2ef8e37\") " pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:09.067790 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.067746 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-48xg4" Sep 19 09:07:09.122904 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.122869 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:09.123062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.122921 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:09.123062 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.123012 2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:07:09.123062 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.123041 2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:07:09.123201 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.123074 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs podName:3f5f3742-e891-4188-aa1d-c48c5e8e8a0b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:41.123060686 +0000 UTC m=+67.881181689 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs") pod "network-metrics-daemon-nst5s" (UID: "3f5f3742-e891-4188-aa1d-c48c5e8e8a0b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:07:09.123201 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.123137 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret podName:6f74b8f0-71dd-482c-aa43-b65acd2cb16b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:41.123120675 +0000 UTC m=+67.881241692 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret") pod "global-pull-secret-syncer-fljvj" (UID: "6f74b8f0-71dd-482c-aa43-b65acd2cb16b") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:07:09.224177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.223997 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:09.224358 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.224162 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:07:09.224358 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.224241 2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:07:09.224358 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.224257 2585 projected.go:196] Error preparing data for projected volume kube-api-access-2wzv4 for pod openshift-network-diagnostics/network-check-target-gml7f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:07:09.224358 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.224335 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4 podName:fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:41.224317725 +0000 UTC m=+67.982438741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2wzv4" (UniqueName: "kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4") pod "network-check-target-gml7f" (UID: "fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:07:09.397529 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.397491 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-48xg4"] Sep 19 09:07:09.399003 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:09.398974 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40b15ed_37e4_476a_a915_ae88b2ef8e37.slice/crio-281f4063b5a10ad9a5704d0a3f250435c69f5f81c447ce20378c88f2668cf37f WatchSource:0}: Error finding container 281f4063b5a10ad9a5704d0a3f250435c69f5f81c447ce20378c88f2668cf37f: Status 404 returned error can't find the container with id 281f4063b5a10ad9a5704d0a3f250435c69f5f81c447ce20378c88f2668cf37f Sep 19 09:07:09.426456 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.426385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:09.426628 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.426534 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:07:09.426628 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:09.426596 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls podName:7cf31f40-1983-41a0-8bdd-88249a000156 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:10.426579832 +0000 UTC m=+37.184700835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls") pod "dns-default-zvr64" (UID: "7cf31f40-1983-41a0-8bdd-88249a000156") : secret "dns-default-metrics-tls" not found Sep 19 09:07:09.430423 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.430404 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:09.431035 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.430406 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:09.434485 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.434463 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 09:07:09.435274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.435254 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 19 09:07:09.435412 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.435392 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 19 09:07:09.435462 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.435257 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2r86\"" Sep 19 09:07:09.436152 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.436129 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fszt9\"" Sep 19 09:07:09.719578 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:09.719449 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-48xg4" event={"ID":"c40b15ed-37e4-476a-a915-ae88b2ef8e37","Type":"ContainerStarted","Data":"281f4063b5a10ad9a5704d0a3f250435c69f5f81c447ce20378c88f2668cf37f"} Sep 19 09:07:10.430058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:10.430021 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:10.433298 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:10.433278 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 09:07:10.434400 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:10.434380 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:10.434576 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:10.434550 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:07:10.434672 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:10.434626 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls podName:7cf31f40-1983-41a0-8bdd-88249a000156 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:12.434602535 +0000 UTC m=+39.192723551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls") pod "dns-default-zvr64" (UID: "7cf31f40-1983-41a0-8bdd-88249a000156") : secret "dns-default-metrics-tls" not found Sep 19 09:07:11.728360 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:11.728322 2585 generic.go:356] "Generic (PLEG): container finished" podID="76ea5338-fc9a-45a1-9111-a727e59130fa" containerID="87e88b6c0a5f0889fdc3ac5159fb5838ba68735643afe5ddfb636f6626b6ca0e" exitCode=0 Sep 19 09:07:11.728895 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:11.728374 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerDied","Data":"87e88b6c0a5f0889fdc3ac5159fb5838ba68735643afe5ddfb636f6626b6ca0e"} Sep 19 09:07:12.452810 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:12.452767 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:12.452989 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:12.452918 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:07:12.452989 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:12.452988 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls podName:7cf31f40-1983-41a0-8bdd-88249a000156 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:16.452968317 +0000 UTC m=+43.211089324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls") pod "dns-default-zvr64" (UID: "7cf31f40-1983-41a0-8bdd-88249a000156") : secret "dns-default-metrics-tls" not found Sep 19 09:07:12.735121 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:12.735021 2585 generic.go:356] "Generic (PLEG): container finished" podID="76ea5338-fc9a-45a1-9111-a727e59130fa" containerID="963b4ec46751f08fddc0ba2d548d80a4a1d700037851d48948b673f7b8eeaaf9" exitCode=0 Sep 19 09:07:12.735121 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:12.735101 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerDied","Data":"963b4ec46751f08fddc0ba2d548d80a4a1d700037851d48948b673f7b8eeaaf9"} Sep 19 09:07:13.739253 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:13.739048 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-48xg4" event={"ID":"c40b15ed-37e4-476a-a915-ae88b2ef8e37","Type":"ContainerStarted","Data":"e37ea527b78bb05aad3e72915da85a5302167c07ff466d41dc03af560bc3dbc0"} Sep 19 09:07:13.742371 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:13.742344 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtr4" event={"ID":"76ea5338-fc9a-45a1-9111-a727e59130fa","Type":"ContainerStarted","Data":"20b6eaddcf9591bbb868940067cdbd89c7c73d2370efd1036fec5a564f55ebcd"} Sep 19 09:07:13.766723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:13.766677 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-48xg4" podStartSLOduration=2.355523306 podStartE2EDuration="5.766661246s" totalImagesPullingTime="3.41113794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:13.765636351 +0000 UTC m=+40.523757375" watchObservedRunningTime="2026-09-19 09:07:13.766661246 +0000 UTC m=+40.524782271" Sep 19 09:07:13.798099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:13.798043 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-chtr4" podStartSLOduration=10.742326094 podStartE2EDuration="38.798024319s" totalImagesPullingTime="28.055698225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:13.797308019 +0000 UTC m=+40.555429046" watchObservedRunningTime="2026-09-19 09:07:13.798024319 +0000 UTC m=+40.556145343" Sep 19 09:07:16.481201 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:16.481162 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:16.481685 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:16.481308 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:07:16.481685 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:16.481370 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls podName:7cf31f40-1983-41a0-8bdd-88249a000156 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:24.481353324 +0000 UTC m=+51.239474327 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls") pod "dns-default-zvr64" (UID: "7cf31f40-1983-41a0-8bdd-88249a000156") : secret "dns-default-metrics-tls" not found Sep 19 09:07:17.699808 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.699772 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n"] Sep 19 09:07:17.703655 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.703635 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.711495 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.711476 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 19 09:07:17.712122 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.712090 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-56b8994dc8-sp2rh"] Sep 19 09:07:17.712225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.712164 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 19 09:07:17.712225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.712166 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 19 09:07:17.712225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.712166 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:07:17.712225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.712194 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xb2k7\"" Sep 19 09:07:17.715508 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.715492 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.718109 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.718081 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 19 09:07:17.718219 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.718198 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 19 09:07:17.718296 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.718236 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 19 09:07:17.718452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.718434 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-797nq\"" Sep 19 09:07:17.718550 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.718436 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 19 09:07:17.720926 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.720904 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n"] Sep 19 09:07:17.722289 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.722272 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 19 09:07:17.722540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.722506 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 19 09:07:17.740841 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.740808 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56b8994dc8-sp2rh"] Sep 19 09:07:17.835714 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.835683 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-kx6t7"] Sep 19 09:07:17.839159 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.839141 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:17.843279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.843259 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 19 09:07:17.843507 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.843483 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 19 09:07:17.843631 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.843486 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 19 09:07:17.843631 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.843537 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:07:17.845668 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.845653 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gtpd5\"" Sep 19 09:07:17.857830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.857809 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 19 09:07:17.869285 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.869262 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv"] Sep 19 09:07:17.872667 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.872654 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:17.876061 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.876048 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 19 09:07:17.876412 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.876399 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:07:17.876624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.876605 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-m9vxw\"" Sep 19 09:07:17.876701 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.876674 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 19 09:07:17.890958 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.890934 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-default-certificate\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.891056 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.890963 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-stats-auth\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.891056 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.890994 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-serving-cert\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.891164 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.891150 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.891266 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.891180 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqqbx\" (UniqueName: \"kubernetes.io/projected/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-kube-api-access-gqqbx\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.891266 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.891205 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-config\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.891266 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.891242 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.891376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.891265 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc8f8\" (UniqueName: \"kubernetes.io/projected/f1c28be5-7f78-42b1-9126-fda3c48b9e04-kube-api-access-tc8f8\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.900705 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.900682 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv"] Sep 19 09:07:17.914006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.913983 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt"] Sep 19 09:07:17.917303 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.917284 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" Sep 19 09:07:17.919660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.919642 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fvjwm\"" Sep 19 09:07:17.928365 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.928346 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-kx6t7"] Sep 19 09:07:17.936118 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.936098 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt"] Sep 19 09:07:17.981346 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.981289 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-689ddbdd68-s97hq"] Sep 19 09:07:17.984626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.984612 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:17.988271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.988255 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 19 09:07:17.988271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.988270 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 19 09:07:17.988570 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.988555 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 19 09:07:17.988641 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.988627 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2nnxz\"" Sep 19 09:07:17.991555 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.991536 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-default-certificate\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.991677 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.991561 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-stats-auth\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.991677 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.991586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-serving-cert\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.991677 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.991614 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxnzb\" (UniqueName: \"kubernetes.io/projected/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-kube-api-access-jxnzb\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:17.991838 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.991737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-config\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:17.992171 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992151 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-serving-cert\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:17.992268 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992234 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.992315 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992276 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-trusted-ca\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:17.992362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992308 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqqbx\" (UniqueName: \"kubernetes.io/projected/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-kube-api-access-gqqbx\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.992362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992332 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:17.992444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992369 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-config\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.992444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992393 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvw2s\" (UniqueName: \"kubernetes.io/projected/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-kube-api-access-nvw2s\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:17.992444 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:17.992430 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:18.492409106 +0000 UTC m=+45.250530112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : configmap references non-existent config key: service-ca.crt Sep 19 09:07:17.992594 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992460 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.992594 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.992494 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc8f8\" (UniqueName: \"kubernetes.io/projected/f1c28be5-7f78-42b1-9126-fda3c48b9e04-kube-api-access-tc8f8\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.993208 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:17.992970 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 09:07:17.993208 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:17.993018 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:18.493005099 +0000 UTC m=+45.251126106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : secret "router-metrics-certs-default" not found Sep 19 09:07:17.993208 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.993169 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-config\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.995741 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.995723 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-serving-cert\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:17.995958 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.995942 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-stats-auth\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:17.996017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:17.996000 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-default-certificate\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:18.000553 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.000533 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 19 09:07:18.013468 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.013441 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqqbx\" (UniqueName: \"kubernetes.io/projected/eaea39f1-a910-46d2-9ac1-3ecffd526b4c-kube-api-access-gqqbx\") pod \"service-ca-operator-57cdc4b768-rwq7n\" (UID: \"eaea39f1-a910-46d2-9ac1-3ecffd526b4c\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:18.014434 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.014408 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-689ddbdd68-s97hq"] Sep 19 09:07:18.014835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.014818 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc8f8\" (UniqueName: \"kubernetes.io/projected/f1c28be5-7f78-42b1-9126-fda3c48b9e04-kube-api-access-tc8f8\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:18.093363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093329 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-serving-cert\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.093553 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093386 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n64nr\" (UniqueName: \"kubernetes.io/projected/99517554-a063-4c80-8120-481d0dc23633-kube-api-access-n64nr\") pod \"network-check-source-7cbc45fddc-cz8gt\" (UID: \"99517554-a063-4c80-8120-481d0dc23633\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" Sep 19 09:07:18.093553 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093493 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-certificates\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.093553 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093540 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvw2s\" (UniqueName: \"kubernetes.io/projected/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-kube-api-access-nvw2s\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.093681 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093591 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxnzb\" (UniqueName: \"kubernetes.io/projected/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-kube-api-access-jxnzb\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:18.093681 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-config\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.093681 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093670 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qd8h\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-kube-api-access-6qd8h\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.093818 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093713 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-trusted-ca\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.093888 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093867 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-trusted-ca\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.093939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.093924 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-installation-pull-secrets\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.094028 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.094011 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:18.094071 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.094047 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-ca-trust-extracted\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.094125 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.094080 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.094170 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.094142 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 19 09:07:18.094218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.094174 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-image-registry-private-configuration\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.094265 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.094217 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls podName:9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:18.594196394 +0000 UTC m=+45.352317400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-r9ktv" (UID: "9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f") : secret "samples-operator-tls" not found Sep 19 09:07:18.094265 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.094240 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-bound-sa-token\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.097752 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.094880 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-config\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.097752 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.095361 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-trusted-ca\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.098423 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.098406 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-serving-cert\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.127894 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.127861 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvw2s\" (UniqueName: \"kubernetes.io/projected/fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26-kube-api-access-nvw2s\") pod \"console-operator-fd4bb46c-kx6t7\" (UID: \"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26\") " pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.134903 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.134879 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxnzb\" (UniqueName: \"kubernetes.io/projected/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-kube-api-access-jxnzb\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:18.147808 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.147789 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:18.195716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.195665 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-image-registry-private-configuration\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.195887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.195719 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-bound-sa-token\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.195887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.195782 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n64nr\" (UniqueName: \"kubernetes.io/projected/99517554-a063-4c80-8120-481d0dc23633-kube-api-access-n64nr\") pod \"network-check-source-7cbc45fddc-cz8gt\" (UID: \"99517554-a063-4c80-8120-481d0dc23633\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" Sep 19 09:07:18.195887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.195811 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-certificates\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.196745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.196482 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-certificates\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.196745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.196551 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qd8h\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-kube-api-access-6qd8h\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.196745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.196594 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-trusted-ca\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.196745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.196618 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-installation-pull-secrets\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.196745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.196664 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-ca-trust-extracted\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.196745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.196691 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.197684 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.197663 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-ca-trust-extracted\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.197900 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.197747 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-trusted-ca\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.198039 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.197769 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:07:18.198039 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.197929 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689ddbdd68-s97hq: secret "image-registry-tls" not found Sep 19 09:07:18.198039 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.198010 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls podName:8398fa8f-39a8-46f6-931a-d9c53dad5bfb nodeName:}" failed. No retries permitted until 2026-09-19 09:07:18.697992062 +0000 UTC m=+45.456113170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls") pod "image-registry-689ddbdd68-s97hq" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb") : secret "image-registry-tls" not found Sep 19 09:07:18.199875 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.199773 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-image-registry-private-configuration\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.200204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.200181 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-installation-pull-secrets\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.222460 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.222436 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-bound-sa-token\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.223124 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.223102 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qd8h\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-kube-api-access-6qd8h\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.224583 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.224566 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n64nr\" (UniqueName: \"kubernetes.io/projected/99517554-a063-4c80-8120-481d0dc23633-kube-api-access-n64nr\") pod \"network-check-source-7cbc45fddc-cz8gt\" (UID: \"99517554-a063-4c80-8120-481d0dc23633\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" Sep 19 09:07:18.225320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.225307 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" Sep 19 09:07:18.312406 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.312378 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" Sep 19 09:07:18.452011 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.451982 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-kx6t7"] Sep 19 09:07:18.455071 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:18.455047 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7d6fe0_eeee_4a99_a9be_c83ee47ccd26.slice/crio-c0341b8954bc2c968ca2159924efc72d111398914a0791d09171a615a9d09436 WatchSource:0}: Error finding container c0341b8954bc2c968ca2159924efc72d111398914a0791d09171a615a9d09436: Status 404 returned error can't find the container with id c0341b8954bc2c968ca2159924efc72d111398914a0791d09171a615a9d09436 Sep 19 09:07:18.499860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.499785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:18.499860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.499852 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:18.500019 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.499935 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 09:07:18.500019 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.499965 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:19.499949349 +0000 UTC m=+46.258070356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : configmap references non-existent config key: service-ca.crt Sep 19 09:07:18.500019 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.500000 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:19.499986571 +0000 UTC m=+46.258107597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : secret "router-metrics-certs-default" not found Sep 19 09:07:18.511422 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.511395 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt"] Sep 19 09:07:18.512275 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:18.512252 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99517554_a063_4c80_8120_481d0dc23633.slice/crio-6b0c092062cffbdbf98c26eb2cb1fb031a4ba28d44d68c824767be57fb83ab8a WatchSource:0}: Error finding container 6b0c092062cffbdbf98c26eb2cb1fb031a4ba28d44d68c824767be57fb83ab8a: Status 404 returned error can't find the container with id 6b0c092062cffbdbf98c26eb2cb1fb031a4ba28d44d68c824767be57fb83ab8a Sep 19 09:07:18.592084 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.592060 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n"] Sep 19 09:07:18.594289 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:18.594262 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaea39f1_a910_46d2_9ac1_3ecffd526b4c.slice/crio-27aca270cdf2753c35fd064a4178e96f20d8a908eb96955833a2923f3809c1fa WatchSource:0}: Error finding container 27aca270cdf2753c35fd064a4178e96f20d8a908eb96955833a2923f3809c1fa: Status 404 returned error can't find the container with id 27aca270cdf2753c35fd064a4178e96f20d8a908eb96955833a2923f3809c1fa Sep 19 09:07:18.600833 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.600808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:18.600962 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.600946 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 19 09:07:18.601010 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.601002 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls podName:9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:19.600986383 +0000 UTC m=+46.359107387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-r9ktv" (UID: "9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f") : secret "samples-operator-tls" not found Sep 19 09:07:18.701840 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.701801 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:18.702236 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.701955 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:07:18.702236 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.701973 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689ddbdd68-s97hq: secret "image-registry-tls" not found Sep 19 09:07:18.702236 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:18.702030 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls podName:8398fa8f-39a8-46f6-931a-d9c53dad5bfb nodeName:}" failed. No retries permitted until 2026-09-19 09:07:19.702009093 +0000 UTC m=+46.460130110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls") pod "image-registry-689ddbdd68-s97hq" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb") : secret "image-registry-tls" not found Sep 19 09:07:18.756095 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.755995 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" event={"ID":"99517554-a063-4c80-8120-481d0dc23633","Type":"ContainerStarted","Data":"6b0c092062cffbdbf98c26eb2cb1fb031a4ba28d44d68c824767be57fb83ab8a"} Sep 19 09:07:18.757080 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.757057 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" event={"ID":"eaea39f1-a910-46d2-9ac1-3ecffd526b4c","Type":"ContainerStarted","Data":"27aca270cdf2753c35fd064a4178e96f20d8a908eb96955833a2923f3809c1fa"} Sep 19 09:07:18.758134 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:18.758108 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" event={"ID":"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26","Type":"ContainerStarted","Data":"c0341b8954bc2c968ca2159924efc72d111398914a0791d09171a615a9d09436"} Sep 19 09:07:19.510428 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:19.509843 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:19.510428 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:19.509918 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:19.510428 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.510010 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 09:07:19.510428 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.510070 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:21.510055986 +0000 UTC m=+48.268176989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : configmap references non-existent config key: service-ca.crt Sep 19 09:07:19.510428 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.510091 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:21.510081934 +0000 UTC m=+48.268202938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : secret "router-metrics-certs-default" not found Sep 19 09:07:19.611311 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:19.611273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:19.611565 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.611549 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 19 09:07:19.611633 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.611619 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls podName:9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:21.611599194 +0000 UTC m=+48.369720220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-r9ktv" (UID: "9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f") : secret "samples-operator-tls" not found Sep 19 09:07:19.712583 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:19.712536 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:19.713064 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.712843 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:07:19.713064 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.712862 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689ddbdd68-s97hq: secret "image-registry-tls" not found Sep 19 09:07:19.713064 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:19.713029 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls podName:8398fa8f-39a8-46f6-931a-d9c53dad5bfb nodeName:}" failed. No retries permitted until 2026-09-19 09:07:21.713008226 +0000 UTC m=+48.471129235 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls") pod "image-registry-689ddbdd68-s97hq" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb") : secret "image-registry-tls" not found Sep 19 09:07:21.529639 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:21.529603 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:21.530183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:21.529720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:21.530183 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.529790 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:25.529773244 +0000 UTC m=+52.287894250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : configmap references non-existent config key: service-ca.crt Sep 19 09:07:21.530183 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.529911 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 09:07:21.530183 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.529960 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:25.529948517 +0000 UTC m=+52.288069533 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : secret "router-metrics-certs-default" not found Sep 19 09:07:21.630410 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:21.630374 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:21.630616 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.630496 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 19 09:07:21.630616 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.630574 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls podName:9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:25.630560246 +0000 UTC m=+52.388681285 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-r9ktv" (UID: "9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f") : secret "samples-operator-tls" not found Sep 19 09:07:21.731392 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:21.731360 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:21.731588 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.731511 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:07:21.731588 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.731538 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689ddbdd68-s97hq: secret "image-registry-tls" not found Sep 19 09:07:21.731706 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:21.731591 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls podName:8398fa8f-39a8-46f6-931a-d9c53dad5bfb nodeName:}" failed. No retries permitted until 2026-09-19 09:07:25.731576945 +0000 UTC m=+52.489697967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls") pod "image-registry-689ddbdd68-s97hq" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb") : secret "image-registry-tls" not found Sep 19 09:07:22.986184 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:22.986151 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2wb9f"] Sep 19 09:07:23.012431 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.012406 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.021757 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.020436 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 19 09:07:23.022372 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.022071 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 19 09:07:23.024992 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.023240 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w276g\"" Sep 19 09:07:23.024992 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.023582 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 19 09:07:23.025693 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.025672 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 19 09:07:23.026310 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.026288 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2wb9f"] Sep 19 09:07:23.040310 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.040285 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.040431 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.040330 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.040490 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.040438 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f43266-21f8-4171-9548-09b85d106b6f-data-volume\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.040560 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.040488 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hmzd\" (UniqueName: \"kubernetes.io/projected/26f43266-21f8-4171-9548-09b85d106b6f-kube-api-access-2hmzd\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.040620 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.040563 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/26f43266-21f8-4171-9548-09b85d106b6f-crio-socket\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141424 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.141381 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f43266-21f8-4171-9548-09b85d106b6f-data-volume\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141598 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.141480 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hmzd\" (UniqueName: \"kubernetes.io/projected/26f43266-21f8-4171-9548-09b85d106b6f-kube-api-access-2hmzd\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141598 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.141545 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/26f43266-21f8-4171-9548-09b85d106b6f-crio-socket\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141598 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.141586 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141713 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.141681 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/26f43266-21f8-4171-9548-09b85d106b6f-crio-socket\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141713 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.141696 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:07:23.141787 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.141722 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141787 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.141769 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:23.641748485 +0000 UTC m=+50.399869492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : secret "insights-runtime-extractor-tls" not found Sep 19 09:07:23.141882 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.141790 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f43266-21f8-4171-9548-09b85d106b6f-data-volume\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.141882 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.141810 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:07:23.141882 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.141861 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:23.641844279 +0000 UTC m=+50.399965303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : configmap "kube-rbac-proxy" not found Sep 19 09:07:23.156216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.156190 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hmzd\" (UniqueName: \"kubernetes.io/projected/26f43266-21f8-4171-9548-09b85d106b6f-kube-api-access-2hmzd\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.646964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.646925 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.647147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.646984 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:23.647147 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.647089 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:07:23.647147 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.647106 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:07:23.647246 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.647155 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:24.647135971 +0000 UTC m=+51.405256980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : configmap "kube-rbac-proxy" not found Sep 19 09:07:23.647246 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:23.647173 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:24.647164194 +0000 UTC m=+51.405285203 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : secret "insights-runtime-extractor-tls" not found Sep 19 09:07:23.773069 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.773028 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" event={"ID":"99517554-a063-4c80-8120-481d0dc23633","Type":"ContainerStarted","Data":"3f19f95c2cd72ef767e1d67b36b21cb75c3f42fe90e53f16fc60f0bff3e4da5d"} Sep 19 09:07:23.774802 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.774771 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" event={"ID":"eaea39f1-a910-46d2-9ac1-3ecffd526b4c","Type":"ContainerStarted","Data":"4b974330f815ff20d4705f1b816acf28db6fb4c9d770d0a95910a5d3d5970a35"} Sep 19 09:07:23.776422 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.776404 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/0.log" Sep 19 09:07:23.776567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.776436 2585 generic.go:356] "Generic (PLEG): container finished" podID="fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26" containerID="d9647c60a9e819b59dd98c639be00b0a0128e5e7df8f9b37e5833105cedd5201" exitCode=255 Sep 19 09:07:23.776567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.776473 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" event={"ID":"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26","Type":"ContainerDied","Data":"d9647c60a9e819b59dd98c639be00b0a0128e5e7df8f9b37e5833105cedd5201"} Sep 19 09:07:23.776728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.776710 2585 scope.go:117] "RemoveContainer" containerID="d9647c60a9e819b59dd98c639be00b0a0128e5e7df8f9b37e5833105cedd5201" Sep 19 09:07:23.836379 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.836330 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-cz8gt" podStartSLOduration=2.629742929 podStartE2EDuration="6.836313031s" totalImagesPullingTime="4.206570102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:23.801980404 +0000 UTC m=+50.560101441" watchObservedRunningTime="2026-09-19 09:07:23.836313031 +0000 UTC m=+50.594434055" Sep 19 09:07:23.870961 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:23.870909 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" podStartSLOduration=2.755044442 podStartE2EDuration="6.870885735s" totalImagesPullingTime="4.115841293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:23.836439181 +0000 UTC m=+50.594560206" watchObservedRunningTime="2026-09-19 09:07:23.870885735 +0000 UTC m=+50.629006761" Sep 19 09:07:24.554764 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.554727 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:24.555138 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:24.554854 2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:07:24.555138 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:24.554908 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls podName:7cf31f40-1983-41a0-8bdd-88249a000156 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:40.554893362 +0000 UTC m=+67.313014384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls") pod "dns-default-zvr64" (UID: "7cf31f40-1983-41a0-8bdd-88249a000156") : secret "dns-default-metrics-tls" not found Sep 19 09:07:24.655576 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.655547 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:24.655739 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.655589 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:24.655739 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:24.655683 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:07:24.655739 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:24.655705 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:07:24.655739 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:24.655733 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:26.655721151 +0000 UTC m=+53.413842167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : configmap "kube-rbac-proxy" not found Sep 19 09:07:24.655866 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:24.655756 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:26.655744021 +0000 UTC m=+53.413865042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : secret "insights-runtime-extractor-tls" not found Sep 19 09:07:24.781155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.781131 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:07:24.781564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.781550 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/0.log" Sep 19 09:07:24.781640 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.781585 2585 generic.go:356] "Generic (PLEG): container finished" podID="fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26" containerID="195a83de9484aeab5c8e9cb0c245c16c2880379a5efabce4314e34fe02b3ff22" exitCode=255 Sep 19 09:07:24.781677 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.781642 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" event={"ID":"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26","Type":"ContainerDied","Data":"195a83de9484aeab5c8e9cb0c245c16c2880379a5efabce4314e34fe02b3ff22"} Sep 19 09:07:24.781710 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.781698 2585 scope.go:117] "RemoveContainer" containerID="d9647c60a9e819b59dd98c639be00b0a0128e5e7df8f9b37e5833105cedd5201" Sep 19 09:07:24.781860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:24.781837 2585 scope.go:117] "RemoveContainer" containerID="195a83de9484aeab5c8e9cb0c245c16c2880379a5efabce4314e34fe02b3ff22" Sep 19 09:07:24.782037 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:24.782016 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-kx6t7_openshift-console-operator(fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26)\"" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" podUID="fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26" Sep 19 09:07:25.392385 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:25.392359 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wcqst_2f96adac-d4cf-4684-b8cf-2a87800cbf10/dns-node-resolver/0.log" Sep 19 09:07:25.562910 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:25.562870 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:25.563369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:25.562951 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:25.563369 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.563008 2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 19 09:07:25.563369 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.563070 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.563054518 +0000 UTC m=+60.321175552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : secret "router-metrics-certs-default" not found Sep 19 09:07:25.563369 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.563084 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.563078042 +0000 UTC m=+60.321199045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : configmap references non-existent config key: service-ca.crt Sep 19 09:07:25.664333 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:25.664251 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:25.664484 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.664390 2585 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 19 09:07:25.664484 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.664448 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls podName:9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.664435225 +0000 UTC m=+60.422556228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-r9ktv" (UID: "9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f") : secret "samples-operator-tls" not found Sep 19 09:07:25.765180 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:25.765146 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:25.765328 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.765283 2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:07:25.765328 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.765296 2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-689ddbdd68-s97hq: secret "image-registry-tls" not found Sep 19 09:07:25.765417 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.765340 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls podName:8398fa8f-39a8-46f6-931a-d9c53dad5bfb nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.765327428 +0000 UTC m=+60.523448431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls") pod "image-registry-689ddbdd68-s97hq" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb") : secret "image-registry-tls" not found Sep 19 09:07:25.785716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:25.785689 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:07:25.786038 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:25.786020 2585 scope.go:117] "RemoveContainer" containerID="195a83de9484aeab5c8e9cb0c245c16c2880379a5efabce4314e34fe02b3ff22" Sep 19 09:07:25.786220 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:25.786200 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-kx6t7_openshift-console-operator(fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26)\"" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" podUID="fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26" Sep 19 09:07:26.180694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.180668 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kghq6_c1e4bdae-16ac-478a-be9d-b108434edc8b/node-ca/0.log" Sep 19 09:07:26.672864 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.672829 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:26.673280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.672877 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:26.673280 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:26.672975 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:07:26.673280 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:26.673016 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:07:26.673280 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:26.673037 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:30.673022395 +0000 UTC m=+57.431143398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : secret "insights-runtime-extractor-tls" not found Sep 19 09:07:26.673280 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:26.673064 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:30.673052533 +0000 UTC m=+57.431173542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : configmap "kube-rbac-proxy" not found Sep 19 09:07:26.846064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.846035 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-72s6w"] Sep 19 09:07:26.862607 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.862589 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.865158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.865132 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48xg4_c40b15ed-37e4-476a-a915-ae88b2ef8e37/serve-healthcheck-canary/0.log" Sep 19 09:07:26.868278 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.868255 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 19 09:07:26.868411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.868386 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 19 09:07:26.868411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.868386 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 19 09:07:26.872315 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.872299 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6m8jz\"" Sep 19 09:07:26.872452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.872437 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 19 09:07:26.874328 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.874310 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-signing-key\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.874465 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.874451 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-signing-cabundle\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.874577 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.874561 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbt9f\" (UniqueName: \"kubernetes.io/projected/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-kube-api-access-qbt9f\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.893468 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.893444 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-72s6w"] Sep 19 09:07:26.975440 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.975410 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-signing-cabundle\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.975581 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.975469 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbt9f\" (UniqueName: \"kubernetes.io/projected/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-kube-api-access-qbt9f\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.975581 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.975505 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-signing-key\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.976122 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.976105 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-signing-cabundle\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.977901 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.977880 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-signing-key\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:26.996041 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:26.996014 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbt9f\" (UniqueName: \"kubernetes.io/projected/1545308c-b1c5-4baa-89f0-8fcec99c3f6a-kube-api-access-qbt9f\") pod \"service-ca-55f6fc7b86-72s6w\" (UID: \"1545308c-b1c5-4baa-89f0-8fcec99c3f6a\") " pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:27.170774 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:27.170693 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" Sep 19 09:07:27.474224 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:27.474142 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-72s6w"] Sep 19 09:07:27.477687 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:27.477658 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1545308c_b1c5_4baa_89f0_8fcec99c3f6a.slice/crio-5665eb8baff99c48d3afcdda4787435d8bc8fe5633da456927be21a11172ce7f WatchSource:0}: Error finding container 5665eb8baff99c48d3afcdda4787435d8bc8fe5633da456927be21a11172ce7f: Status 404 returned error can't find the container with id 5665eb8baff99c48d3afcdda4787435d8bc8fe5633da456927be21a11172ce7f Sep 19 09:07:27.794056 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:27.793975 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" event={"ID":"1545308c-b1c5-4baa-89f0-8fcec99c3f6a","Type":"ContainerStarted","Data":"fb0b07f8b0edab374512d5417c9440e5066c4bf1e1c30716169f5473ad85fe46"} Sep 19 09:07:27.794056 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:27.794023 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" event={"ID":"1545308c-b1c5-4baa-89f0-8fcec99c3f6a","Type":"ContainerStarted","Data":"5665eb8baff99c48d3afcdda4787435d8bc8fe5633da456927be21a11172ce7f"} Sep 19 09:07:27.836352 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:27.836301 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-72s6w" podStartSLOduration=1.836286255 podStartE2EDuration="1.836286255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:27.83568906 +0000 UTC m=+54.593810084" watchObservedRunningTime="2026-09-19 09:07:27.836286255 +0000 UTC m=+54.594407281" Sep 19 09:07:28.148777 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:28.148738 2585 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:28.148978 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:28.148806 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:28.149192 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:28.149179 2585 scope.go:117] "RemoveContainer" containerID="195a83de9484aeab5c8e9cb0c245c16c2880379a5efabce4314e34fe02b3ff22" Sep 19 09:07:28.149392 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:28.149371 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-kx6t7_openshift-console-operator(fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26)\"" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" podUID="fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26" Sep 19 09:07:30.704933 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:30.704895 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:30.705296 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:30.704945 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:30.705296 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:30.705048 2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:07:30.705296 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:30.705078 2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:07:30.705296 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:30.705111 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:38.705096571 +0000 UTC m=+65.463217587 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : secret "insights-runtime-extractor-tls" not found Sep 19 09:07:30.705296 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:30.705127 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm podName:26f43266-21f8-4171-9548-09b85d106b6f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:38.705120984 +0000 UTC m=+65.463241987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2wb9f" (UID: "26f43266-21f8-4171-9548-09b85d106b6f") : configmap "kube-rbac-proxy" not found Sep 19 09:07:33.625102 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.625072 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:33.625656 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.625136 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:33.625656 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:07:33.625290 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle podName:f1c28be5-7f78-42b1-9126-fda3c48b9e04 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:49.625271577 +0000 UTC m=+76.383392598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle") pod "router-default-56b8994dc8-sp2rh" (UID: "f1c28be5-7f78-42b1-9126-fda3c48b9e04") : configmap references non-existent config key: service-ca.crt Sep 19 09:07:33.627368 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.627345 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1c28be5-7f78-42b1-9126-fda3c48b9e04-metrics-certs\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:33.725697 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.725667 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:33.727924 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.727905 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-r9ktv\" (UID: \"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:33.780450 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.780418 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" Sep 19 09:07:33.827482 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.827142 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:33.830574 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.830549 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"image-registry-689ddbdd68-s97hq\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:33.893722 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:33.893640 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:34.080832 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:34.080800 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv"] Sep 19 09:07:34.183903 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:34.183815 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-689ddbdd68-s97hq"] Sep 19 09:07:34.188127 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:34.188101 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8398fa8f_39a8_46f6_931a_d9c53dad5bfb.slice/crio-a74639f714e952e993e4b1530dc2663e84f625b2f5e3ebde7c06b459e411d926 WatchSource:0}: Error finding container a74639f714e952e993e4b1530dc2663e84f625b2f5e3ebde7c06b459e411d926: Status 404 returned error can't find the container with id a74639f714e952e993e4b1530dc2663e84f625b2f5e3ebde7c06b459e411d926 Sep 19 09:07:34.717293 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:34.717264 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6c442" Sep 19 09:07:34.819031 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:34.818996 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" event={"ID":"8398fa8f-39a8-46f6-931a-d9c53dad5bfb","Type":"ContainerStarted","Data":"a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08"} Sep 19 09:07:34.819193 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:34.819051 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:07:34.819193 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:34.819071 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" event={"ID":"8398fa8f-39a8-46f6-931a-d9c53dad5bfb","Type":"ContainerStarted","Data":"a74639f714e952e993e4b1530dc2663e84f625b2f5e3ebde7c06b459e411d926"} Sep 19 09:07:34.820384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:34.820358 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" event={"ID":"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f","Type":"ContainerStarted","Data":"5f14e8ee04410bebe8904b2293ef45de664139ef40db1c9c161ef26425c23834"} Sep 19 09:07:35.826307 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:35.826274 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" event={"ID":"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f","Type":"ContainerStarted","Data":"e0538e801b040dfe8e14fcedcd0210f24893fbb6f7317e97cb8e77387479f785"} Sep 19 09:07:35.826659 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:35.826369 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" event={"ID":"9b6bf4cf-2956-4f7c-a099-5a4a09e1ec9f","Type":"ContainerStarted","Data":"652f3712106f6c839303a0e0952f460a400dba84676c0bf2b3dd6e86e515ef97"} Sep 19 09:07:35.865019 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:35.864977 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" podStartSLOduration=18.864962002 podStartE2EDuration="18.864962002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:34.856790296 +0000 UTC m=+61.614911323" watchObservedRunningTime="2026-09-19 09:07:35.864962002 +0000 UTC m=+62.623083030" Sep 19 09:07:35.865240 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:35.865220 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-r9ktv" podStartSLOduration=17.288346785 podStartE2EDuration="18.865215665s" totalImagesPullingTime="1.57686888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:35.863921044 +0000 UTC m=+62.622042069" watchObservedRunningTime="2026-09-19 09:07:35.865215665 +0000 UTC m=+62.623336683" Sep 19 09:07:38.765399 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:38.765364 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:38.765863 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:38.765408 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:38.765940 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:38.765873 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/26f43266-21f8-4171-9548-09b85d106b6f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:38.767772 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:38.767747 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/26f43266-21f8-4171-9548-09b85d106b6f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2wb9f\" (UID: \"26f43266-21f8-4171-9548-09b85d106b6f\") " pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:38.926746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:38.926723 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w276g\"" Sep 19 09:07:38.934876 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:38.934859 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2wb9f" Sep 19 09:07:39.272695 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:39.272664 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2wb9f"] Sep 19 09:07:39.275914 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:39.275886 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f43266_21f8_4171_9548_09b85d106b6f.slice/crio-97defa347d6651c6d6882ced413d6bac1154b3612016f5baae8a6c427dbf6149 WatchSource:0}: Error finding container 97defa347d6651c6d6882ced413d6bac1154b3612016f5baae8a6c427dbf6149: Status 404 returned error can't find the container with id 97defa347d6651c6d6882ced413d6bac1154b3612016f5baae8a6c427dbf6149 Sep 19 09:07:39.843090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:39.843060 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wb9f" event={"ID":"26f43266-21f8-4171-9548-09b85d106b6f","Type":"ContainerStarted","Data":"18dd02dbaa7f6266d105d0928e9fe15c4c02baaba6e0809aae5d29342e660fd7"} Sep 19 09:07:39.843416 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:39.843134 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wb9f" event={"ID":"26f43266-21f8-4171-9548-09b85d106b6f","Type":"ContainerStarted","Data":"97defa347d6651c6d6882ced413d6bac1154b3612016f5baae8a6c427dbf6149"} Sep 19 09:07:40.578380 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:40.578332 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:40.581006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:40.580974 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7cf31f40-1983-41a0-8bdd-88249a000156-metrics-tls\") pod \"dns-default-zvr64\" (UID: \"7cf31f40-1983-41a0-8bdd-88249a000156\") " pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:40.848499 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:40.848416 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wb9f" event={"ID":"26f43266-21f8-4171-9548-09b85d106b6f","Type":"ContainerStarted","Data":"b6bc78e789fcb0d51af941a9327f42d57d5ad656ee3c08f7fb95ce3b091a0a8c"} Sep 19 09:07:40.850096 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:40.850070 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xchdv\"" Sep 19 09:07:40.857646 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:40.857627 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:41.140367 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.140296 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvr64"] Sep 19 09:07:41.183472 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.183436 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:41.183644 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.183485 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:41.187171 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.187151 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 09:07:41.188446 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.188427 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 09:07:41.196789 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.196757 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6f74b8f0-71dd-482c-aa43-b65acd2cb16b-original-pull-secret\") pod \"global-pull-secret-syncer-fljvj\" (UID: \"6f74b8f0-71dd-482c-aa43-b65acd2cb16b\") " pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:41.196903 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.196827 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5f3742-e891-4188-aa1d-c48c5e8e8a0b-metrics-certs\") pod \"network-metrics-daemon-nst5s\" (UID: \"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b\") " pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:41.245097 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.245072 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fszt9\"" Sep 19 09:07:41.252322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.252289 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nst5s" Sep 19 09:07:41.255583 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:41.255556 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf31f40_1983_41a0_8bdd_88249a000156.slice/crio-a3d297773f98c5c5134aaf8cfa9e12c94331d8db02bac007315d2f7b6ac89899 WatchSource:0}: Error finding container a3d297773f98c5c5134aaf8cfa9e12c94331d8db02bac007315d2f7b6ac89899: Status 404 returned error can't find the container with id a3d297773f98c5c5134aaf8cfa9e12c94331d8db02bac007315d2f7b6ac89899 Sep 19 09:07:41.284377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.284352 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:41.286865 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.286840 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wzv4\" (UniqueName: \"kubernetes.io/projected/fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0-kube-api-access-2wzv4\") pod \"network-check-target-gml7f\" (UID: \"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0\") " pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:41.339960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.339939 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fljvj" Sep 19 09:07:41.538558 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.538473 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nst5s"] Sep 19 09:07:41.541570 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:41.541548 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5f3742_e891_4188_aa1d_c48c5e8e8a0b.slice/crio-b6673b3c9bf33ff49a0f8582c31af4e4eac04014989de99c79482672a1296b35 WatchSource:0}: Error finding container b6673b3c9bf33ff49a0f8582c31af4e4eac04014989de99c79482672a1296b35: Status 404 returned error can't find the container with id b6673b3c9bf33ff49a0f8582c31af4e4eac04014989de99c79482672a1296b35 Sep 19 09:07:41.552901 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.552882 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2r86\"" Sep 19 09:07:41.558104 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.558087 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:41.615492 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.615422 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fljvj"] Sep 19 09:07:41.619280 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:41.619251 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f74b8f0_71dd_482c_aa43_b65acd2cb16b.slice/crio-690aec79a0efcfa017f89356761f1c48a92f6acbaabd2d08b9a573578e908215 WatchSource:0}: Error finding container 690aec79a0efcfa017f89356761f1c48a92f6acbaabd2d08b9a573578e908215: Status 404 returned error can't find the container with id 690aec79a0efcfa017f89356761f1c48a92f6acbaabd2d08b9a573578e908215 Sep 19 09:07:41.840745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.840707 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gml7f"] Sep 19 09:07:41.844414 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:41.844385 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa52dafb_6bb7_4c95_8a89_1cc38aa1e4a0.slice/crio-7d396716566ace22f674f3a11303d9032b33327cfdddada66ad06c15a203b39d WatchSource:0}: Error finding container 7d396716566ace22f674f3a11303d9032b33327cfdddada66ad06c15a203b39d: Status 404 returned error can't find the container with id 7d396716566ace22f674f3a11303d9032b33327cfdddada66ad06c15a203b39d Sep 19 09:07:41.853688 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.853659 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2wb9f" event={"ID":"26f43266-21f8-4171-9548-09b85d106b6f","Type":"ContainerStarted","Data":"4ee1de9224138250c487df5cfe4ed671d6afa81d7fa462fef86ce2910875ff7f"} Sep 19 09:07:41.855155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.855122 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fljvj" event={"ID":"6f74b8f0-71dd-482c-aa43-b65acd2cb16b","Type":"ContainerStarted","Data":"690aec79a0efcfa017f89356761f1c48a92f6acbaabd2d08b9a573578e908215"} Sep 19 09:07:41.856482 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.856373 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gml7f" event={"ID":"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0","Type":"ContainerStarted","Data":"7d396716566ace22f674f3a11303d9032b33327cfdddada66ad06c15a203b39d"} Sep 19 09:07:41.857634 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.857608 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nst5s" event={"ID":"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b","Type":"ContainerStarted","Data":"b6673b3c9bf33ff49a0f8582c31af4e4eac04014989de99c79482672a1296b35"} Sep 19 09:07:41.858792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.858769 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvr64" event={"ID":"7cf31f40-1983-41a0-8bdd-88249a000156","Type":"ContainerStarted","Data":"a3d297773f98c5c5134aaf8cfa9e12c94331d8db02bac007315d2f7b6ac89899"} Sep 19 09:07:41.883555 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:41.883341 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2wb9f" podStartSLOduration=17.978105298 podStartE2EDuration="19.883318924s" totalImagesPullingTime="1.905213626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:07:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:41.883263184 +0000 UTC m=+68.641384208" watchObservedRunningTime="2026-09-19 09:07:41.883318924 +0000 UTC m=+68.641439949" Sep 19 09:07:42.864816 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:42.864775 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gml7f" event={"ID":"fa52dafb-6bb7-4c95-8a89-1cc38aa1e4a0","Type":"ContainerStarted","Data":"e98f564bce71d609d3d8a18b1f1754d219e9dd8288d9fcb1b5bdd9eb940fca9e"} Sep 19 09:07:42.865305 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:42.865069 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:07:42.893929 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:42.893359 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gml7f" podStartSLOduration=67.893338105 podStartE2EDuration="1m7.893338105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:42.892504606 +0000 UTC m=+69.650625633" watchObservedRunningTime="2026-09-19 09:07:42.893338105 +0000 UTC m=+69.651459128" Sep 19 09:07:43.430293 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.430226 2585 scope.go:117] "RemoveContainer" containerID="195a83de9484aeab5c8e9cb0c245c16c2880379a5efabce4314e34fe02b3ff22" Sep 19 09:07:43.872385 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.872353 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:07:43.872900 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.872459 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" event={"ID":"fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26","Type":"ContainerStarted","Data":"309b2e6cadfbaf2bc7a6007e1c989db503c43d7bfaa0a10d0ed8d302d636689a"} Sep 19 09:07:43.872900 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.872870 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:43.875384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.875349 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nst5s" event={"ID":"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b","Type":"ContainerStarted","Data":"6b6d28e3fe7a72a0204b3793b693ae202ced5e8d1f10d53a04c181280f83206d"} Sep 19 09:07:43.875557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.875406 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nst5s" event={"ID":"3f5f3742-e891-4188-aa1d-c48c5e8e8a0b","Type":"ContainerStarted","Data":"f96f23c80197ee847239091a64d4f3b24d15e36901aa942d8d85f095d036620b"} Sep 19 09:07:43.878545 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.878469 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvr64" event={"ID":"7cf31f40-1983-41a0-8bdd-88249a000156","Type":"ContainerStarted","Data":"ba36bdf9629d0ed393141baa87497c4a973777a8b7f9fcd5479864e054761667"} Sep 19 09:07:43.878631 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.878545 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvr64" event={"ID":"7cf31f40-1983-41a0-8bdd-88249a000156","Type":"ContainerStarted","Data":"a9bc1d296aada7da567b9cd86f905d1573ab4a3f6e0a3840cd6763e524104ba4"} Sep 19 09:07:43.903684 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.903633 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" podStartSLOduration=22.649360129 podStartE2EDuration="26.903615276s" totalImagesPullingTime="4.254255147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:43.903206251 +0000 UTC m=+70.661327278" watchObservedRunningTime="2026-09-19 09:07:43.903615276 +0000 UTC m=+70.661736304" Sep 19 09:07:43.948001 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.947951 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nst5s" podStartSLOduration=67.364930603 podStartE2EDuration="1m8.947933084s" totalImagesPullingTime="1.583002481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:43.94726479 +0000 UTC m=+70.705385815" watchObservedRunningTime="2026-09-19 09:07:43.947933084 +0000 UTC m=+70.706054109" Sep 19 09:07:43.979202 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:43.979074 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zvr64" podStartSLOduration=34.108965249 podStartE2EDuration="35.979063123s" totalImagesPullingTime="1.870097874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:43.979010544 +0000 UTC m=+70.737131569" watchObservedRunningTime="2026-09-19 09:07:43.979063123 +0000 UTC m=+70.737184148" Sep 19 09:07:44.547114 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:44.547084 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-kx6t7" Sep 19 09:07:44.883419 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:44.883386 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:46.893820 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:46.893789 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fljvj" event={"ID":"6f74b8f0-71dd-482c-aa43-b65acd2cb16b","Type":"ContainerStarted","Data":"72dc4aab4e70b544fd56611d349e2175e3ac3c02077b0dbf91b4e4fd56a6d2a5"} Sep 19 09:07:46.928512 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:46.928458 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fljvj" podStartSLOduration=67.724941976 podStartE2EDuration="1m11.928442414s" totalImagesPullingTime="4.203500438s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:46.928071602 +0000 UTC m=+73.686192626" watchObservedRunningTime="2026-09-19 09:07:46.928442414 +0000 UTC m=+73.686563439" Sep 19 09:07:49.657815 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:49.657774 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:49.658334 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:49.658315 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1c28be5-7f78-42b1-9126-fda3c48b9e04-service-ca-bundle\") pod \"router-default-56b8994dc8-sp2rh\" (UID: \"f1c28be5-7f78-42b1-9126-fda3c48b9e04\") " pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:49.826273 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:49.826243 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-797nq\"" Sep 19 09:07:49.834504 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:49.834483 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:50.134592 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:50.134553 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56b8994dc8-sp2rh"] Sep 19 09:07:50.139540 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:07:50.139493 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c28be5_7f78_42b1_9126_fda3c48b9e04.slice/crio-a577fcec4e8ac875a1cfcc93ddd86f8fcd41d1381c7306662b2ec451fcc13c7b WatchSource:0}: Error finding container a577fcec4e8ac875a1cfcc93ddd86f8fcd41d1381c7306662b2ec451fcc13c7b: Status 404 returned error can't find the container with id a577fcec4e8ac875a1cfcc93ddd86f8fcd41d1381c7306662b2ec451fcc13c7b Sep 19 09:07:50.908387 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:50.908350 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" event={"ID":"f1c28be5-7f78-42b1-9126-fda3c48b9e04","Type":"ContainerStarted","Data":"34c03bd13889fa8af5a5314a1d395d729696e60a0082c657bfabb234e0cb0416"} Sep 19 09:07:50.908759 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:50.908399 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" event={"ID":"f1c28be5-7f78-42b1-9126-fda3c48b9e04","Type":"ContainerStarted","Data":"a577fcec4e8ac875a1cfcc93ddd86f8fcd41d1381c7306662b2ec451fcc13c7b"} Sep 19 09:07:50.945798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:50.945747 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podStartSLOduration=33.945730375 podStartE2EDuration="33.945730375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:50.944824372 +0000 UTC m=+77.702945397" watchObservedRunningTime="2026-09-19 09:07:50.945730375 +0000 UTC m=+77.703851399" Sep 19 09:07:51.835534 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:51.835491 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:51.838046 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:51.838020 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:51.838046 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:51.838046 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:51.838046 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:51.838252 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:51.838076 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:52.836312 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:52.836273 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:52.836312 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:52.836312 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:52.836312 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:52.836752 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:52.836341 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:53.836506 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:53.836470 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:53.836506 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:53.836506 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:53.836506 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:53.837033 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:53.836543 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:53.898100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:53.898067 2585 patch_prober.go:28] interesting pod/image-registry-689ddbdd68-s97hq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 19 09:07:53.898243 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:53.898124 2585 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 09:07:54.835943 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:54.835909 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:54.835943 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:54.835943 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:54.835943 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:54.836187 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:54.835974 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:54.888471 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:54.888441 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zvr64" Sep 19 09:07:55.830207 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:55.830177 2585 patch_prober.go:28] interesting pod/image-registry-689ddbdd68-s97hq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 19 09:07:55.830366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:55.830231 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 09:07:55.835617 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:55.835596 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:55.835617 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:55.835617 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:55.835617 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:55.835765 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:55.835637 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:56.836316 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:56.836279 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:56.836316 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:56.836316 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:56.836316 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:56.836788 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:56.836339 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:57.836241 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:57.836210 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:57.836241 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:57.836241 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:57.836241 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:57.836707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:57.836264 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:58.835630 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:58.835595 2585 patch_prober.go:28] interesting pod/router-default-56b8994dc8-sp2rh container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Sep 19 09:07:58.835630 ip-10-0-132-57 kubenswrapper[2585]: [-]has-synced failed: reason withheld Sep 19 09:07:58.835630 ip-10-0-132-57 kubenswrapper[2585]: [+]process-running ok Sep 19 09:07:58.835630 ip-10-0-132-57 kubenswrapper[2585]: healthz check failed Sep 19 09:07:58.835944 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:58.835653 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" podUID="f1c28be5-7f78-42b1-9126-fda3c48b9e04" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 19 09:07:59.834943 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:59.834892 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:59.836190 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:59.836170 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:07:59.943101 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:07:59.943075 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-56b8994dc8-sp2rh" Sep 19 09:08:03.897770 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:03.897737 2585 patch_prober.go:28] interesting pod/image-registry-689ddbdd68-s97hq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 19 09:08:03.898230 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:03.897795 2585 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 09:08:04.780045 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.780014 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4"] Sep 19 09:08:04.784705 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.784681 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:04.799781 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.799747 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 19 09:08:04.802395 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.800818 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 19 09:08:04.802395 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.802173 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-mfvwl\"" Sep 19 09:08:04.803433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.803203 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 19 09:08:04.803433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.803379 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 19 09:08:04.837346 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.837313 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4"] Sep 19 09:08:04.865342 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.865309 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2042331-5730-43f8-b156-848936bba475-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4\" (UID: \"e2042331-5730-43f8-b156-848936bba475\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:04.865513 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.865372 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvfzg\" (UniqueName: \"kubernetes.io/projected/e2042331-5730-43f8-b156-848936bba475-kube-api-access-mvfzg\") pod \"managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4\" (UID: \"e2042331-5730-43f8-b156-848936bba475\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:04.966145 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.966111 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2042331-5730-43f8-b156-848936bba475-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4\" (UID: \"e2042331-5730-43f8-b156-848936bba475\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:04.966634 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.966180 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvfzg\" (UniqueName: \"kubernetes.io/projected/e2042331-5730-43f8-b156-848936bba475-kube-api-access-mvfzg\") pod \"managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4\" (UID: \"e2042331-5730-43f8-b156-848936bba475\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:04.968750 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:04.968728 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2042331-5730-43f8-b156-848936bba475-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4\" (UID: \"e2042331-5730-43f8-b156-848936bba475\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:05.031667 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.031583 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78"] Sep 19 09:08:05.034759 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.034742 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.038161 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.038136 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 19 09:08:05.051394 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.051370 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k"] Sep 19 09:08:05.052868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.052839 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvfzg\" (UniqueName: \"kubernetes.io/projected/e2042331-5730-43f8-b156-848936bba475-kube-api-access-mvfzg\") pod \"managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4\" (UID: \"e2042331-5730-43f8-b156-848936bba475\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:05.054350 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.054332 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" Sep 19 09:08:05.077530 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.077491 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-c7pnh\"" Sep 19 09:08:05.081911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.081888 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 19 09:08:05.102915 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.102886 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" Sep 19 09:08:05.154482 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.154415 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78"] Sep 19 09:08:05.156060 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.155577 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-689ddbdd68-s97hq"] Sep 19 09:08:05.161071 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.161052 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:08:05.168327 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.168296 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plmwg\" (UniqueName: \"kubernetes.io/projected/90b25b23-1569-4947-b5cb-41b21c57ce08-kube-api-access-plmwg\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.168452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.168345 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b25b23-1569-4947-b5cb-41b21c57ce08-tmp\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.168452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.168389 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/90b25b23-1569-4947-b5cb-41b21c57ce08-klusterlet-config\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.168577 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.168477 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/33bbb5a8-1c66-474f-a849-ed184d180eef-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-mzl8k\" (UID: \"33bbb5a8-1c66-474f-a849-ed184d180eef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" Sep 19 09:08:05.172610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.172579 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-f5l9p"] Sep 19 09:08:05.177679 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.177658 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-f5l9p" Sep 19 09:08:05.181477 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.181458 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 19 09:08:05.181742 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.181727 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fg75w\"" Sep 19 09:08:05.188650 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.188633 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 19 09:08:05.189173 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.189155 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bdbcd9569-sspfs"] Sep 19 09:08:05.192112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.192096 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.196610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.196594 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ljgnv\"" Sep 19 09:08:05.197455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.197439 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 09:08:05.197681 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.197666 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 09:08:05.197941 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.197929 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 09:08:05.212823 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.212800 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 09:08:05.213031 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.213016 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 09:08:05.213891 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.213869 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k"] Sep 19 09:08:05.228710 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.228685 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 09:08:05.252085 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.252054 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bdbcd9569-sspfs"] Sep 19 09:08:05.253030 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.253004 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-f5l9p"] Sep 19 09:08:05.269330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269303 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/33bbb5a8-1c66-474f-a849-ed184d180eef-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-mzl8k\" (UID: \"33bbb5a8-1c66-474f-a849-ed184d180eef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" Sep 19 09:08:05.269443 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269344 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plmwg\" (UniqueName: \"kubernetes.io/projected/90b25b23-1569-4947-b5cb-41b21c57ce08-kube-api-access-plmwg\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.269443 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269377 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b25b23-1569-4947-b5cb-41b21c57ce08-tmp\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.269443 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269435 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-service-ca\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.269573 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269482 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/90b25b23-1569-4947-b5cb-41b21c57ce08-klusterlet-config\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.269573 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269509 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-serving-cert\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.269675 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269662 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-oauth-config\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.269728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269692 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvnrx\" (UniqueName: \"kubernetes.io/projected/55c3e3b3-a08d-4145-bc84-31733b991710-kube-api-access-zvnrx\") pod \"downloads-6594f4c75-f5l9p\" (UID: \"55c3e3b3-a08d-4145-bc84-31733b991710\") " pod="openshift-console/downloads-6594f4c75-f5l9p" Sep 19 09:08:05.269785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269764 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-config\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.269785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269776 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90b25b23-1569-4947-b5cb-41b21c57ce08-tmp\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.269851 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269796 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-trusted-ca-bundle\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.269851 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269829 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-oauth-serving-cert\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.269927 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.269860 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc48l\" (UniqueName: \"kubernetes.io/projected/401b2b78-c57c-4ef2-937d-fa3582fac38d-kube-api-access-vc48l\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.271812 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.271792 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/33bbb5a8-1c66-474f-a849-ed184d180eef-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-mzl8k\" (UID: \"33bbb5a8-1c66-474f-a849-ed184d180eef\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" Sep 19 09:08:05.271899 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.271890 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/90b25b23-1569-4947-b5cb-41b21c57ce08-klusterlet-config\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.290419 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.290352 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b7768f645-vgq4t"] Sep 19 09:08:05.293562 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.293544 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.327674 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.327642 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b7768f645-vgq4t"] Sep 19 09:08:05.341728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.341700 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plmwg\" (UniqueName: \"kubernetes.io/projected/90b25b23-1569-4947-b5cb-41b21c57ce08-kube-api-access-plmwg\") pod \"klusterlet-addon-workmgr-77dfdf5c9c-f9l78\" (UID: \"90b25b23-1569-4947-b5cb-41b21c57ce08\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.343502 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.343480 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:05.366774 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.366705 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" Sep 19 09:08:05.370285 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370232 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-registry-tls\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.370285 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370281 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33361761-7682-4d9f-8cba-8954d27deb57-registry-certificates\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.370381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370301 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-bound-sa-token\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.370381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370325 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-service-ca\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.370476 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370452 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33361761-7682-4d9f-8cba-8954d27deb57-image-registry-private-configuration\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.370558 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370490 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33361761-7682-4d9f-8cba-8954d27deb57-installation-pull-secrets\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.370621 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370574 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx7v4\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-kube-api-access-lx7v4\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.370673 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370628 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33361761-7682-4d9f-8cba-8954d27deb57-trusted-ca\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.370673 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370657 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-serving-cert\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.370770 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370697 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-oauth-config\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.370886 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370865 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvnrx\" (UniqueName: \"kubernetes.io/projected/55c3e3b3-a08d-4145-bc84-31733b991710-kube-api-access-zvnrx\") pod \"downloads-6594f4c75-f5l9p\" (UID: \"55c3e3b3-a08d-4145-bc84-31733b991710\") " pod="openshift-console/downloads-6594f4c75-f5l9p" Sep 19 09:08:05.370958 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370926 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-config\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.371010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370954 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-trusted-ca-bundle\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.371010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.370981 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-oauth-serving-cert\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.371108 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.371011 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33361761-7682-4d9f-8cba-8954d27deb57-ca-trust-extracted\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.371108 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.371043 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc48l\" (UniqueName: \"kubernetes.io/projected/401b2b78-c57c-4ef2-937d-fa3582fac38d-kube-api-access-vc48l\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.371203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.371173 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-service-ca\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.371781 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.371757 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-config\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.371931 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.371832 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-oauth-serving-cert\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.372223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.372186 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-trusted-ca-bundle\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.373514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.373489 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-serving-cert\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.373620 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.373556 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-oauth-config\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.397138 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.396921 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvnrx\" (UniqueName: \"kubernetes.io/projected/55c3e3b3-a08d-4145-bc84-31733b991710-kube-api-access-zvnrx\") pod \"downloads-6594f4c75-f5l9p\" (UID: \"55c3e3b3-a08d-4145-bc84-31733b991710\") " pod="openshift-console/downloads-6594f4c75-f5l9p" Sep 19 09:08:05.398920 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.398831 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc48l\" (UniqueName: \"kubernetes.io/projected/401b2b78-c57c-4ef2-937d-fa3582fac38d-kube-api-access-vc48l\") pod \"console-5bdbcd9569-sspfs\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.454014 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.453969 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4"] Sep 19 09:08:05.458163 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:08:05.458115 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2042331_5730_43f8_b156_848936bba475.slice/crio-c7d30f726182e390ec33c452af7f8406098ce9957a01be2fed2414e1b12074f8 WatchSource:0}: Error finding container c7d30f726182e390ec33c452af7f8406098ce9957a01be2fed2414e1b12074f8: Status 404 returned error can't find the container with id c7d30f726182e390ec33c452af7f8406098ce9957a01be2fed2414e1b12074f8 Sep 19 09:08:05.471439 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471411 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33361761-7682-4d9f-8cba-8954d27deb57-registry-certificates\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.471439 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471442 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-bound-sa-token\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.471660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33361761-7682-4d9f-8cba-8954d27deb57-image-registry-private-configuration\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.471660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471554 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33361761-7682-4d9f-8cba-8954d27deb57-installation-pull-secrets\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.471660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471594 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx7v4\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-kube-api-access-lx7v4\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.471660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471641 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33361761-7682-4d9f-8cba-8954d27deb57-trusted-ca\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.471793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471719 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33361761-7682-4d9f-8cba-8954d27deb57-ca-trust-extracted\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.471793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.471758 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-registry-tls\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.472281 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.472235 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/33361761-7682-4d9f-8cba-8954d27deb57-ca-trust-extracted\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.472446 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.472414 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/33361761-7682-4d9f-8cba-8954d27deb57-registry-certificates\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.473047 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.473024 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/33361761-7682-4d9f-8cba-8954d27deb57-trusted-ca\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.474429 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.474389 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/33361761-7682-4d9f-8cba-8954d27deb57-installation-pull-secrets\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.474637 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.474612 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/33361761-7682-4d9f-8cba-8954d27deb57-image-registry-private-configuration\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.474737 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.474612 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-registry-tls\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.487170 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.487126 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-f5l9p" Sep 19 09:08:05.500751 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.500722 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:05.554455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.554405 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-bound-sa-token\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.558973 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.558947 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx7v4\" (UniqueName: \"kubernetes.io/projected/33361761-7682-4d9f-8cba-8954d27deb57-kube-api-access-lx7v4\") pod \"image-registry-5b7768f645-vgq4t\" (UID: \"33361761-7682-4d9f-8cba-8954d27deb57\") " pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.603087 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.603050 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:05.725890 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.725748 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78"] Sep 19 09:08:05.728385 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:08:05.728359 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b25b23_1569_4947_b5cb_41b21c57ce08.slice/crio-3311d8d8050defa1eac6f1a2206268cecf8c99e7ed40cbf75e298109a796a0e5 WatchSource:0}: Error finding container 3311d8d8050defa1eac6f1a2206268cecf8c99e7ed40cbf75e298109a796a0e5: Status 404 returned error can't find the container with id 3311d8d8050defa1eac6f1a2206268cecf8c99e7ed40cbf75e298109a796a0e5 Sep 19 09:08:05.751697 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.751662 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k"] Sep 19 09:08:05.751697 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:08:05.751682 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bbb5a8_1c66_474f_a849_ed184d180eef.slice/crio-2b610826963c499a85b2ec921c749a40842f0fc43613766d3a7041eb9c840cac WatchSource:0}: Error finding container 2b610826963c499a85b2ec921c749a40842f0fc43613766d3a7041eb9c840cac: Status 404 returned error can't find the container with id 2b610826963c499a85b2ec921c749a40842f0fc43613766d3a7041eb9c840cac Sep 19 09:08:05.823113 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.823078 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-f5l9p"] Sep 19 09:08:05.825876 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:08:05.825848 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c3e3b3_a08d_4145_bc84_31733b991710.slice/crio-8bd780001380a6ffa8b65efdf37a7c41c9cca69940ac858c8c844bf3fd277c7a WatchSource:0}: Error finding container 8bd780001380a6ffa8b65efdf37a7c41c9cca69940ac858c8c844bf3fd277c7a: Status 404 returned error can't find the container with id 8bd780001380a6ffa8b65efdf37a7c41c9cca69940ac858c8c844bf3fd277c7a Sep 19 09:08:05.873085 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.873059 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bdbcd9569-sspfs"] Sep 19 09:08:05.874804 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:08:05.874777 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401b2b78_c57c_4ef2_937d_fa3582fac38d.slice/crio-a6b3f05080051a1f2d4847522ab5d260422053dd07343581c85d524fdc0e854d WatchSource:0}: Error finding container a6b3f05080051a1f2d4847522ab5d260422053dd07343581c85d524fdc0e854d: Status 404 returned error can't find the container with id a6b3f05080051a1f2d4847522ab5d260422053dd07343581c85d524fdc0e854d Sep 19 09:08:05.963679 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.963644 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bdbcd9569-sspfs" event={"ID":"401b2b78-c57c-4ef2-937d-fa3582fac38d","Type":"ContainerStarted","Data":"a6b3f05080051a1f2d4847522ab5d260422053dd07343581c85d524fdc0e854d"} Sep 19 09:08:05.965058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.965021 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" event={"ID":"33bbb5a8-1c66-474f-a849-ed184d180eef","Type":"ContainerStarted","Data":"2b610826963c499a85b2ec921c749a40842f0fc43613766d3a7041eb9c840cac"} Sep 19 09:08:05.966351 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.966323 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" event={"ID":"e2042331-5730-43f8-b156-848936bba475","Type":"ContainerStarted","Data":"c7d30f726182e390ec33c452af7f8406098ce9957a01be2fed2414e1b12074f8"} Sep 19 09:08:05.967783 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.967753 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b7768f645-vgq4t"] Sep 19 09:08:05.967959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.967923 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-f5l9p" event={"ID":"55c3e3b3-a08d-4145-bc84-31733b991710","Type":"ContainerStarted","Data":"8bd780001380a6ffa8b65efdf37a7c41c9cca69940ac858c8c844bf3fd277c7a"} Sep 19 09:08:05.969154 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:05.969128 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" event={"ID":"90b25b23-1569-4947-b5cb-41b21c57ce08","Type":"ContainerStarted","Data":"3311d8d8050defa1eac6f1a2206268cecf8c99e7ed40cbf75e298109a796a0e5"} Sep 19 09:08:05.973013 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:08:05.972977 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33361761_7682_4d9f_8cba_8954d27deb57.slice/crio-6939d2c87ec74eeb16388904d0981e33f1ef412f42e8f2c694c2c6f42654d3e7 WatchSource:0}: Error finding container 6939d2c87ec74eeb16388904d0981e33f1ef412f42e8f2c694c2c6f42654d3e7: Status 404 returned error can't find the container with id 6939d2c87ec74eeb16388904d0981e33f1ef412f42e8f2c694c2c6f42654d3e7 Sep 19 09:08:06.988596 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:06.988353 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" event={"ID":"33361761-7682-4d9f-8cba-8954d27deb57","Type":"ContainerStarted","Data":"b12cc50439db4a1f6b07515c7302ba0fc6170d975eea4a748543331c85c22bae"} Sep 19 09:08:06.988596 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:06.988424 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" event={"ID":"33361761-7682-4d9f-8cba-8954d27deb57","Type":"ContainerStarted","Data":"6939d2c87ec74eeb16388904d0981e33f1ef412f42e8f2c694c2c6f42654d3e7"} Sep 19 09:08:06.989133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:06.988673 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:07.050281 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:07.050188 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" podStartSLOduration=2.050164704 podStartE2EDuration="2.050164704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:07.049062592 +0000 UTC m=+93.807183617" watchObservedRunningTime="2026-09-19 09:08:07.050164704 +0000 UTC m=+93.808285726" Sep 19 09:08:13.881064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:13.881016 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gml7f" Sep 19 09:08:15.035146 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.035004 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bdbcd9569-sspfs" event={"ID":"401b2b78-c57c-4ef2-937d-fa3582fac38d","Type":"ContainerStarted","Data":"de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198"} Sep 19 09:08:15.036985 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.036957 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" event={"ID":"33bbb5a8-1c66-474f-a849-ed184d180eef","Type":"ContainerStarted","Data":"ae52826ad33f1b91c61bb3ea561721a5d0ce2f54cb2a3c4420678b3d42710e1a"} Sep 19 09:08:15.037203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.037161 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" Sep 19 09:08:15.038809 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.038766 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" event={"ID":"e2042331-5730-43f8-b156-848936bba475","Type":"ContainerStarted","Data":"a4205ae2f149f73065b1b787b3a931cc62282ef38b9c98b0968dbe3e03dfa40c"} Sep 19 09:08:15.040694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.040673 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" event={"ID":"90b25b23-1569-4947-b5cb-41b21c57ce08","Type":"ContainerStarted","Data":"977c5958b1d1b86224a180bbeb4bf6c3f7e8bf537052d082b6bc36d205930201"} Sep 19 09:08:15.041044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.041018 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:15.042416 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.042398 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" Sep 19 09:08:15.042531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.042455 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" Sep 19 09:08:15.075664 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.075322 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bdbcd9569-sspfs" podStartSLOduration=1.129091169 podStartE2EDuration="10.075298795s" totalImagesPullingTime="8.946207626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:15.074990409 +0000 UTC m=+101.833111435" watchObservedRunningTime="2026-09-19 09:08:15.075298795 +0000 UTC m=+101.833419823" Sep 19 09:08:15.106005 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.105948 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dfc646b-rcsg4" podStartSLOduration=1.788862779 podStartE2EDuration="11.105928742s" totalImagesPullingTime="9.317065963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:15.104197626 +0000 UTC m=+101.862318653" watchObservedRunningTime="2026-09-19 09:08:15.105928742 +0000 UTC m=+101.864049766" Sep 19 09:08:15.180019 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.179967 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77dfdf5c9c-f9l78" podStartSLOduration=2.106376449 podStartE2EDuration="11.179951437s" totalImagesPullingTime="9.073574988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:15.139621161 +0000 UTC m=+101.897742185" watchObservedRunningTime="2026-09-19 09:08:15.179951437 +0000 UTC m=+101.938072465" Sep 19 09:08:15.180176 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.180063 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-mzl8k" podStartSLOduration=2.158369323 podStartE2EDuration="11.180053182s" totalImagesPullingTime="9.021683859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:15.178018192 +0000 UTC m=+101.936139218" watchObservedRunningTime="2026-09-19 09:08:15.180053182 +0000 UTC m=+101.938174208" Sep 19 09:08:15.501217 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.501186 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:15.501365 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.501238 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:15.506975 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:15.506950 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:16.049940 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:16.049902 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:08:22.116383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.116350 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7g8r9"] Sep 19 09:08:22.119566 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.119514 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.119855 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.119904 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd2rs\" (UniqueName: \"kubernetes.io/projected/96cfb1a6-633f-4db0-b70b-762ab4ef9447-kube-api-access-pd2rs\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.119959 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-sys\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.119994 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-metrics-client-ca\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.120021 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-tls\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.120047 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-wtmp\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.120080 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-root\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.120145 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.120272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.120183 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-textfile\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.125408 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.125364 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Sep 19 09:08:22.125978 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.125953 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"node-exporter-7g8r9\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="96cfb1a6-633f-4db0-b70b-762ab4ef9447" pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.126352 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.126320 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" Sep 19 09:08:22.126617 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.126592 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" Sep 19 09:08:22.126694 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.126664 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Sep 19 09:08:22.126748 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.126715 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"metrics-client-ca\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" Sep 19 09:08:22.127120 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.127090 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" Sep 19 09:08:22.132745 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:22.132711 2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-vz84s\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vz84s\"" type="*v1.Secret" Sep 19 09:08:22.221257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221219 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-textfile\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221434 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221295 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-textfile\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221434 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221353 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221434 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221385 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd2rs\" (UniqueName: \"kubernetes.io/projected/96cfb1a6-633f-4db0-b70b-762ab4ef9447-kube-api-access-pd2rs\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221434 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221421 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-sys\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221668 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221484 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-sys\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221721 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221658 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-metrics-client-ca\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221721 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221697 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-tls\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221817 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221722 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-wtmp\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221817 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221747 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-root\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.221817 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221790 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.222010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221877 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-root\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.222010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.221904 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-wtmp\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:22.970451 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.970416 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vz84s\"" Sep 19 09:08:22.978699 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:22.978671 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 19 09:08:23.222076 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.221952 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-kube-rbac-proxy-config: failed to sync secret cache: timed out waiting for the condition Sep 19 09:08:23.222568 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.222108 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-kube-rbac-proxy-config podName:96cfb1a6-633f-4db0-b70b-762ab4ef9447 nodeName:}" failed. No retries permitted until 2026-09-19 09:08:23.722085022 +0000 UTC m=+110.480206025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-kube-rbac-proxy-config" (UniqueName: "kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-kube-rbac-proxy-config") pod "node-exporter-7g8r9" (UID: "96cfb1a6-633f-4db0-b70b-762ab4ef9447") : failed to sync secret cache: timed out waiting for the condition Sep 19 09:08:23.222568 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.221962 2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: failed to sync secret cache: timed out waiting for the condition Sep 19 09:08:23.222568 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.222163 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-tls podName:96cfb1a6-633f-4db0-b70b-762ab4ef9447 nodeName:}" failed. No retries permitted until 2026-09-19 09:08:23.722151643 +0000 UTC m=+110.480272648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-tls") pod "node-exporter-7g8r9" (UID: "96cfb1a6-633f-4db0-b70b-762ab4ef9447") : failed to sync secret cache: timed out waiting for the condition Sep 19 09:08:23.222568 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.221970 2585 configmap.go:193] Couldn't get configMap openshift-monitoring/node-exporter-accelerators-collector-config: failed to sync configmap cache: timed out waiting for the condition Sep 19 09:08:23.222568 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.222497 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-accelerators-collector-config podName:96cfb1a6-633f-4db0-b70b-762ab4ef9447 nodeName:}" failed. No retries permitted until 2026-09-19 09:08:23.722480266 +0000 UTC m=+110.480601276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-accelerators-collector-config" (UniqueName: "kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-accelerators-collector-config") pod "node-exporter-7g8r9" (UID: "96cfb1a6-633f-4db0-b70b-762ab4ef9447") : failed to sync configmap cache: timed out waiting for the condition Sep 19 09:08:23.222568 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.221982 2585 configmap.go:193] Couldn't get configMap openshift-monitoring/metrics-client-ca: failed to sync configmap cache: timed out waiting for the condition Sep 19 09:08:23.222923 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:23.222581 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-metrics-client-ca podName:96cfb1a6-633f-4db0-b70b-762ab4ef9447 nodeName:}" failed. No retries permitted until 2026-09-19 09:08:23.722567545 +0000 UTC m=+110.480688553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-client-ca" (UniqueName: "kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-metrics-client-ca") pod "node-exporter-7g8r9" (UID: "96cfb1a6-633f-4db0-b70b-762ab4ef9447") : failed to sync configmap cache: timed out waiting for the condition Sep 19 09:08:23.320322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.320280 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 19 09:08:23.401157 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.401110 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 19 09:08:23.518025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.517936 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 19 09:08:23.578414 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.578372 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 19 09:08:23.587980 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.587949 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd2rs\" (UniqueName: \"kubernetes.io/projected/96cfb1a6-633f-4db0-b70b-762ab4ef9447-kube-api-access-pd2rs\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.703216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.703183 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 19 09:08:23.735250 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.735215 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-metrics-client-ca\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.735455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.735263 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-tls\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.735455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.735325 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.735455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.735387 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.735951 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.735916 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-metrics-client-ca\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.735951 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.735933 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-accelerators-collector-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.738274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.738119 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.738274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.738212 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/96cfb1a6-633f-4db0-b70b-762ab4ef9447-node-exporter-tls\") pod \"node-exporter-7g8r9\" (UID: \"96cfb1a6-633f-4db0-b70b-762ab4ef9447\") " pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.931467 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:23.931425 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7g8r9" Sep 19 09:08:23.964158 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:08:23.964124 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96cfb1a6_633f_4db0_b70b_762ab4ef9447.slice/crio-7766a729e485ce35a2e505d1fe6d426e80a2237356730720cd486f8db28ccc9c WatchSource:0}: Error finding container 7766a729e485ce35a2e505d1fe6d426e80a2237356730720cd486f8db28ccc9c: Status 404 returned error can't find the container with id 7766a729e485ce35a2e505d1fe6d426e80a2237356730720cd486f8db28ccc9c Sep 19 09:08:24.078064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:24.077957 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-f5l9p" event={"ID":"55c3e3b3-a08d-4145-bc84-31733b991710","Type":"ContainerStarted","Data":"7d5a733927e9133e331addcd877accc3446eb9835b2be2692842939c2e1cf2c6"} Sep 19 09:08:24.078064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:24.078032 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-f5l9p" Sep 19 09:08:24.079550 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:24.079510 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g8r9" event={"ID":"96cfb1a6-633f-4db0-b70b-762ab4ef9447","Type":"ContainerStarted","Data":"7766a729e485ce35a2e505d1fe6d426e80a2237356730720cd486f8db28ccc9c"} Sep 19 09:08:24.079858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:24.079838 2585 patch_prober.go:28] interesting pod/downloads-6594f4c75-f5l9p container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.19:8080/\": dial tcp 10.133.0.19:8080: connect: connection refused" start-of-body= Sep 19 09:08:24.079927 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:24.079884 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6594f4c75-f5l9p" podUID="55c3e3b3-a08d-4145-bc84-31733b991710" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.19:8080/\": dial tcp 10.133.0.19:8080: connect: connection refused" Sep 19 09:08:24.188728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:24.188632 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-f5l9p" podStartSLOduration=1.014720567 podStartE2EDuration="19.188613101s" totalImagesPullingTime="18.173892534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:24.188127004 +0000 UTC m=+110.946248030" watchObservedRunningTime="2026-09-19 09:08:24.188613101 +0000 UTC m=+110.946734139" Sep 19 09:08:25.088160 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:25.087577 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g8r9" event={"ID":"96cfb1a6-633f-4db0-b70b-762ab4ef9447","Type":"ContainerStarted","Data":"876a17e38ce0d39e0e78883d3df58ec9465558244fdbbf65898c84848a6ee46f"} Sep 19 09:08:25.093174 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:25.093124 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-f5l9p" Sep 19 09:08:26.094504 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:26.094468 2585 generic.go:356] "Generic (PLEG): container finished" podID="96cfb1a6-633f-4db0-b70b-762ab4ef9447" containerID="876a17e38ce0d39e0e78883d3df58ec9465558244fdbbf65898c84848a6ee46f" exitCode=0 Sep 19 09:08:26.095032 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:26.094573 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g8r9" event={"ID":"96cfb1a6-633f-4db0-b70b-762ab4ef9447","Type":"ContainerDied","Data":"876a17e38ce0d39e0e78883d3df58ec9465558244fdbbf65898c84848a6ee46f"} Sep 19 09:08:27.102418 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:27.101888 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g8r9" event={"ID":"96cfb1a6-633f-4db0-b70b-762ab4ef9447","Type":"ContainerStarted","Data":"37d0c6bf9dd5cbe7ad136c8febf5f5c7c868075b5986accd5833a0a2e162d08d"} Sep 19 09:08:27.102418 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:27.102005 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7g8r9" event={"ID":"96cfb1a6-633f-4db0-b70b-762ab4ef9447","Type":"ContainerStarted","Data":"796456c527f4eca03b6946748563fbd2fb2e3db48926bd24d0fbbc8dd809e57b"} Sep 19 09:08:27.151595 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:27.151489 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7g8r9" podStartSLOduration=4.086452159 podStartE2EDuration="5.151463925s" totalImagesPullingTime="986.793888ms" totalInitContainerRuntime="78.217878ms" isStatefulPod=true podCreationTimestamp="2026-09-19 09:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:27.149736651 +0000 UTC m=+113.907857690" watchObservedRunningTime="2026-09-19 09:08:27.151463925 +0000 UTC m=+113.909584951" Sep 19 09:08:28.003962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:28.003929 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b7768f645-vgq4t" Sep 19 09:08:30.178620 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.178571 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerName="registry" containerID="cri-o://a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08" gracePeriod=30 Sep 19 09:08:30.505628 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.505593 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:08:30.602140 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602091 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-trusted-ca\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.602354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602166 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-ca-trust-extracted\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.602354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602200 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.602354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602235 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-image-registry-private-configuration\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.602354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602264 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-certificates\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.602354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602296 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-bound-sa-token\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.602354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602325 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qd8h\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-kube-api-access-6qd8h\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.602710 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602362 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-installation-pull-secrets\") pod \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\" (UID: \"8398fa8f-39a8-46f6-931a-d9c53dad5bfb\") " Sep 19 09:08:30.603073 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.602830 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:08:30.603255 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.603228 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:08:30.607293 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.607226 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:08:30.607792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.607687 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:08:30.607792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.607766 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:08:30.607943 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.607791 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:08:30.607943 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.607873 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-kube-api-access-6qd8h" (OuterVolumeSpecName: "kube-api-access-6qd8h") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "kube-api-access-6qd8h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:08:30.613196 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.613160 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8398fa8f-39a8-46f6-931a-d9c53dad5bfb" (UID: "8398fa8f-39a8-46f6-931a-d9c53dad5bfb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:08:30.703771 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703733 2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:30.703953 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703771 2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-image-registry-private-configuration\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:30.703953 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703828 2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-registry-certificates\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:30.703953 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703842 2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-bound-sa-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:30.703953 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703854 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qd8h\" (UniqueName: \"kubernetes.io/projected/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-kube-api-access-6qd8h\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:30.703953 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703863 2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-installation-pull-secrets\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:30.703953 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703873 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-trusted-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:30.703953 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:30.703885 2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8398fa8f-39a8-46f6-931a-d9c53dad5bfb-ca-trust-extracted\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:08:31.126653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.126613 2585 generic.go:356] "Generic (PLEG): container finished" podID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerID="a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08" exitCode=0 Sep 19 09:08:31.126828 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.126667 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" Sep 19 09:08:31.126828 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.126705 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" event={"ID":"8398fa8f-39a8-46f6-931a-d9c53dad5bfb","Type":"ContainerDied","Data":"a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08"} Sep 19 09:08:31.126828 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.126771 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-689ddbdd68-s97hq" event={"ID":"8398fa8f-39a8-46f6-931a-d9c53dad5bfb","Type":"ContainerDied","Data":"a74639f714e952e993e4b1530dc2663e84f625b2f5e3ebde7c06b459e411d926"} Sep 19 09:08:31.126828 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.126799 2585 scope.go:117] "RemoveContainer" containerID="a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08" Sep 19 09:08:31.141508 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.141456 2585 scope.go:117] "RemoveContainer" containerID="a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08" Sep 19 09:08:31.141984 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:08:31.141949 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08\": container with ID starting with a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08 not found: ID does not exist" containerID="a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08" Sep 19 09:08:31.142130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.141993 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08"} err="failed to get container status \"a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08\": rpc error: code = NotFound desc = could not find container \"a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08\": container with ID starting with a0afaa416d65a049404252f8d41b62b1c27b47ab7e6a9fd11f0d58d71c5f5a08 not found: ID does not exist" Sep 19 09:08:31.177138 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.177099 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-689ddbdd68-s97hq"] Sep 19 09:08:31.198327 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.198290 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-689ddbdd68-s97hq"] Sep 19 09:08:31.436209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:31.436121 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" path="/var/lib/kubelet/pods/8398fa8f-39a8-46f6-931a-d9c53dad5bfb/volumes" Sep 19 09:08:37.570446 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:37.570399 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bdbcd9569-sspfs"] Sep 19 09:08:38.974689 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:38.974653 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56b8994dc8-sp2rh_f1c28be5-7f78-42b1-9126-fda3c48b9e04/router/0.log" Sep 19 09:08:39.035279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:39.035252 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48xg4_c40b15ed-37e4-476a-a915-ae88b2ef8e37/serve-healthcheck-canary/0.log" Sep 19 09:08:44.179199 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:44.179160 2585 generic.go:356] "Generic (PLEG): container finished" podID="eaea39f1-a910-46d2-9ac1-3ecffd526b4c" containerID="4b974330f815ff20d4705f1b816acf28db6fb4c9d770d0a95910a5d3d5970a35" exitCode=0 Sep 19 09:08:44.179639 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:44.179205 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" event={"ID":"eaea39f1-a910-46d2-9ac1-3ecffd526b4c","Type":"ContainerDied","Data":"4b974330f815ff20d4705f1b816acf28db6fb4c9d770d0a95910a5d3d5970a35"} Sep 19 09:08:44.179639 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:44.179567 2585 scope.go:117] "RemoveContainer" containerID="4b974330f815ff20d4705f1b816acf28db6fb4c9d770d0a95910a5d3d5970a35" Sep 19 09:08:45.185537 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:08:45.185488 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-rwq7n" event={"ID":"eaea39f1-a910-46d2-9ac1-3ecffd526b4c","Type":"ContainerStarted","Data":"48358f860af438e25c554b65a1ac39d856ca3400f134c6656e50e673d10ca5e6"} Sep 19 09:09:02.589541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.589456 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bdbcd9569-sspfs" podUID="401b2b78-c57c-4ef2-937d-fa3582fac38d" containerName="console" containerID="cri-o://de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198" gracePeriod=15 Sep 19 09:09:02.864403 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.864328 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:09:02.981454 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981419 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc48l\" (UniqueName: \"kubernetes.io/projected/401b2b78-c57c-4ef2-937d-fa3582fac38d-kube-api-access-vc48l\") pod \"401b2b78-c57c-4ef2-937d-fa3582fac38d\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " Sep 19 09:09:02.981682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981485 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-serving-cert\") pod \"401b2b78-c57c-4ef2-937d-fa3582fac38d\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " Sep 19 09:09:02.981682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981549 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-service-ca\") pod \"401b2b78-c57c-4ef2-937d-fa3582fac38d\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " Sep 19 09:09:02.981682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981579 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-oauth-config\") pod \"401b2b78-c57c-4ef2-937d-fa3582fac38d\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " Sep 19 09:09:02.981682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981627 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-oauth-serving-cert\") pod \"401b2b78-c57c-4ef2-937d-fa3582fac38d\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " Sep 19 09:09:02.981682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981671 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-trusted-ca-bundle\") pod \"401b2b78-c57c-4ef2-937d-fa3582fac38d\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " Sep 19 09:09:02.981936 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981707 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-config\") pod \"401b2b78-c57c-4ef2-937d-fa3582fac38d\" (UID: \"401b2b78-c57c-4ef2-937d-fa3582fac38d\") " Sep 19 09:09:02.982025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.981987 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-service-ca" (OuterVolumeSpecName: "service-ca") pod "401b2b78-c57c-4ef2-937d-fa3582fac38d" (UID: "401b2b78-c57c-4ef2-937d-fa3582fac38d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:09:02.982166 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.982134 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "401b2b78-c57c-4ef2-937d-fa3582fac38d" (UID: "401b2b78-c57c-4ef2-937d-fa3582fac38d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:09:02.982230 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.982143 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "401b2b78-c57c-4ef2-937d-fa3582fac38d" (UID: "401b2b78-c57c-4ef2-937d-fa3582fac38d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:09:02.982287 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.982237 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-config" (OuterVolumeSpecName: "console-config") pod "401b2b78-c57c-4ef2-937d-fa3582fac38d" (UID: "401b2b78-c57c-4ef2-937d-fa3582fac38d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:09:02.983883 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.983848 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "401b2b78-c57c-4ef2-937d-fa3582fac38d" (UID: "401b2b78-c57c-4ef2-937d-fa3582fac38d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:09:02.983977 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.983882 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "401b2b78-c57c-4ef2-937d-fa3582fac38d" (UID: "401b2b78-c57c-4ef2-937d-fa3582fac38d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:09:02.983977 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:02.983894 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/401b2b78-c57c-4ef2-937d-fa3582fac38d-kube-api-access-vc48l" (OuterVolumeSpecName: "kube-api-access-vc48l") pod "401b2b78-c57c-4ef2-937d-fa3582fac38d" (UID: "401b2b78-c57c-4ef2-937d-fa3582fac38d"). InnerVolumeSpecName "kube-api-access-vc48l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:09:03.083093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.083058 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc48l\" (UniqueName: \"kubernetes.io/projected/401b2b78-c57c-4ef2-937d-fa3582fac38d-kube-api-access-vc48l\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:09:03.083093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.083088 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:09:03.083093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.083098 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:09:03.083093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.083105 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-oauth-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:09:03.083347 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.083115 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-oauth-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:09:03.083347 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.083124 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-trusted-ca-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:09:03.083347 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.083132 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/401b2b78-c57c-4ef2-937d-fa3582fac38d-console-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:09:03.254531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.254435 2585 generic.go:356] "Generic (PLEG): container finished" podID="401b2b78-c57c-4ef2-937d-fa3582fac38d" containerID="de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198" exitCode=0 Sep 19 09:09:03.254531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.254480 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bdbcd9569-sspfs" Sep 19 09:09:03.254531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.254504 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bdbcd9569-sspfs" event={"ID":"401b2b78-c57c-4ef2-937d-fa3582fac38d","Type":"ContainerDied","Data":"de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198"} Sep 19 09:09:03.254727 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.254571 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bdbcd9569-sspfs" event={"ID":"401b2b78-c57c-4ef2-937d-fa3582fac38d","Type":"ContainerDied","Data":"a6b3f05080051a1f2d4847522ab5d260422053dd07343581c85d524fdc0e854d"} Sep 19 09:09:03.254727 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.254594 2585 scope.go:117] "RemoveContainer" containerID="de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198" Sep 19 09:09:03.263047 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.263031 2585 scope.go:117] "RemoveContainer" containerID="de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198" Sep 19 09:09:03.263302 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:09:03.263282 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198\": container with ID starting with de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198 not found: ID does not exist" containerID="de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198" Sep 19 09:09:03.263346 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.263310 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198"} err="failed to get container status \"de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198\": rpc error: code = NotFound desc = could not find container \"de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198\": container with ID starting with de69de26d729fabe9109be7b0609c93753e34905f95d4637e14363363619b198 not found: ID does not exist" Sep 19 09:09:03.291580 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.291552 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bdbcd9569-sspfs"] Sep 19 09:09:03.298347 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.298318 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bdbcd9569-sspfs"] Sep 19 09:09:03.438737 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:09:03.438705 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="401b2b78-c57c-4ef2-937d-fa3582fac38d" path="/var/lib/kubelet/pods/401b2b78-c57c-4ef2-937d-fa3582fac38d/volumes" Sep 19 09:11:35.370693 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:11:35.370660 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:11:35.372342 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:11:35.372321 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:11:35.374272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:11:35.374249 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:11:35.375785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:11:35.375762 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:11:35.386787 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:11:35.386767 2585 kubelet.go:1648] "Image garbage collection succeeded" Sep 19 09:13:35.639867 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.639835 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f4986c9b6-gxjbk"] Sep 19 09:13:35.640264 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.640098 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="401b2b78-c57c-4ef2-937d-fa3582fac38d" containerName="console" Sep 19 09:13:35.640264 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.640111 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="401b2b78-c57c-4ef2-937d-fa3582fac38d" containerName="console" Sep 19 09:13:35.640264 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.640118 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerName="registry" Sep 19 09:13:35.640264 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.640124 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerName="registry" Sep 19 09:13:35.640264 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.640169 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="401b2b78-c57c-4ef2-937d-fa3582fac38d" containerName="console" Sep 19 09:13:35.640264 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.640176 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8398fa8f-39a8-46f6-931a-d9c53dad5bfb" containerName="registry" Sep 19 09:13:35.641839 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.641814 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.647662 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.647642 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 09:13:35.649927 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.649900 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 09:13:35.650660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.650638 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 09:13:35.650660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.650651 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ljgnv\"" Sep 19 09:13:35.650916 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.650892 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 09:13:35.651102 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.651086 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 09:13:35.660960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.660938 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 09:13:35.666094 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.666072 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f4986c9b6-gxjbk"] Sep 19 09:13:35.733210 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.733184 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-oauth-serving-cert\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.733363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.733227 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-oauth-config\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.733363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.733251 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-serving-cert\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.733363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.733294 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsqf2\" (UniqueName: \"kubernetes.io/projected/495c6114-4f4b-482c-b409-995b97f62002-kube-api-access-rsqf2\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.733363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.733333 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-console-config\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.733363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.733353 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-trusted-ca-bundle\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.733543 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.733378 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-service-ca\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.834682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.834647 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-oauth-serving-cert\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.834850 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.834714 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-oauth-config\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.834850 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.834740 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-serving-cert\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.834850 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.834781 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsqf2\" (UniqueName: \"kubernetes.io/projected/495c6114-4f4b-482c-b409-995b97f62002-kube-api-access-rsqf2\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.834850 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.834815 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-console-config\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.835006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.834917 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-trusted-ca-bundle\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.835006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.834947 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-service-ca\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.835420 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.835392 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-oauth-serving-cert\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.835632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.835604 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-service-ca\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.835847 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.835824 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-trusted-ca-bundle\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.835888 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.835862 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-console-config\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.837048 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.837026 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-oauth-config\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.837189 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.837170 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-serving-cert\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.843873 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.843850 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsqf2\" (UniqueName: \"kubernetes.io/projected/495c6114-4f4b-482c-b409-995b97f62002-kube-api-access-rsqf2\") pod \"console-5f4986c9b6-gxjbk\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:35.953000 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:35.952935 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:36.223865 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:36.223679 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f4986c9b6-gxjbk"] Sep 19 09:13:36.226441 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:13:36.226414 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495c6114_4f4b_482c_b409_995b97f62002.slice/crio-cad30bed60a14ef62caa97cca916e97a52356e494635cee0d391a58901d8106e WatchSource:0}: Error finding container cad30bed60a14ef62caa97cca916e97a52356e494635cee0d391a58901d8106e: Status 404 returned error can't find the container with id cad30bed60a14ef62caa97cca916e97a52356e494635cee0d391a58901d8106e Sep 19 09:13:37.228459 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:37.228421 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4986c9b6-gxjbk" event={"ID":"495c6114-4f4b-482c-b409-995b97f62002","Type":"ContainerStarted","Data":"ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f"} Sep 19 09:13:37.228871 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:37.228475 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4986c9b6-gxjbk" event={"ID":"495c6114-4f4b-482c-b409-995b97f62002","Type":"ContainerStarted","Data":"cad30bed60a14ef62caa97cca916e97a52356e494635cee0d391a58901d8106e"} Sep 19 09:13:37.249857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:37.249802 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f4986c9b6-gxjbk" podStartSLOduration=2.249783959 podStartE2EDuration="2.249783959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:13:37.249643791 +0000 UTC m=+424.007764817" watchObservedRunningTime="2026-09-19 09:13:37.249783959 +0000 UTC m=+424.007904980" Sep 19 09:13:45.953324 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:45.953283 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:45.953806 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:45.953349 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:45.958690 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:45.958659 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:13:46.264243 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:13:46.264141 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:14:44.747577 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.747545 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9f5846bc5-7t25z"] Sep 19 09:14:44.751001 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.750975 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.764363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.764337 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9f5846bc5-7t25z"] Sep 19 09:14:44.841450 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.841418 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-service-ca\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.841450 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.841448 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-serving-cert\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.841678 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.841475 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-console-config\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.841678 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.841492 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-oauth-serving-cert\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.841678 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.841583 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-trusted-ca-bundle\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.841678 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.841624 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lktfk\" (UniqueName: \"kubernetes.io/projected/c3dc326f-0d53-4bb7-868a-db343e04a118-kube-api-access-lktfk\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.841836 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.841696 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-oauth-config\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942043 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-console-config\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942077 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-oauth-serving-cert\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942294 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942105 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-trusted-ca-bundle\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942294 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942130 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lktfk\" (UniqueName: \"kubernetes.io/projected/c3dc326f-0d53-4bb7-868a-db343e04a118-kube-api-access-lktfk\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942294 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942163 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-oauth-config\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942294 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942187 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-service-ca\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942498 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942364 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-serving-cert\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942846 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-console-config\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.942977 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.942956 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-oauth-serving-cert\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.943033 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.943013 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-service-ca\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.943109 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.943093 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-trusted-ca-bundle\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.944919 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.944893 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-oauth-config\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.945006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.944970 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-serving-cert\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:44.950157 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:44.950136 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lktfk\" (UniqueName: \"kubernetes.io/projected/c3dc326f-0d53-4bb7-868a-db343e04a118-kube-api-access-lktfk\") pod \"console-9f5846bc5-7t25z\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:45.060688 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:45.060665 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:45.334082 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:45.334057 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9f5846bc5-7t25z"] Sep 19 09:14:45.336400 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:14:45.336366 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dc326f_0d53_4bb7_868a_db343e04a118.slice/crio-6b105c5dbbc10bfce43d6fc2bff57cd964f1239e81a765f3d6e080a63d1a9f50 WatchSource:0}: Error finding container 6b105c5dbbc10bfce43d6fc2bff57cd964f1239e81a765f3d6e080a63d1a9f50: Status 404 returned error can't find the container with id 6b105c5dbbc10bfce43d6fc2bff57cd964f1239e81a765f3d6e080a63d1a9f50 Sep 19 09:14:45.469983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:45.469951 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5846bc5-7t25z" event={"ID":"c3dc326f-0d53-4bb7-868a-db343e04a118","Type":"ContainerStarted","Data":"6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f"} Sep 19 09:14:45.470175 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:45.470009 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5846bc5-7t25z" event={"ID":"c3dc326f-0d53-4bb7-868a-db343e04a118","Type":"ContainerStarted","Data":"6b105c5dbbc10bfce43d6fc2bff57cd964f1239e81a765f3d6e080a63d1a9f50"} Sep 19 09:14:45.489455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:45.489410 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9f5846bc5-7t25z" podStartSLOduration=1.489394376 podStartE2EDuration="1.489394376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:14:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:14:45.488672038 +0000 UTC m=+492.246793087" watchObservedRunningTime="2026-09-19 09:14:45.489394376 +0000 UTC m=+492.247515422" Sep 19 09:14:55.061082 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:55.061048 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:55.061438 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:55.061106 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:55.065691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:55.065668 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:55.505751 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:55.505682 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:14:55.569398 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:14:55.569370 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f4986c9b6-gxjbk"] Sep 19 09:15:20.589376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.589289 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f4986c9b6-gxjbk" podUID="495c6114-4f4b-482c-b409-995b97f62002" containerName="console" containerID="cri-o://ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f" gracePeriod=15 Sep 19 09:15:20.822476 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.822448 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:15:20.905540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905430 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-oauth-config\") pod \"495c6114-4f4b-482c-b409-995b97f62002\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " Sep 19 09:15:20.905540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905477 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-trusted-ca-bundle\") pod \"495c6114-4f4b-482c-b409-995b97f62002\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " Sep 19 09:15:20.905540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905504 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-service-ca\") pod \"495c6114-4f4b-482c-b409-995b97f62002\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " Sep 19 09:15:20.905799 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905555 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-serving-cert\") pod \"495c6114-4f4b-482c-b409-995b97f62002\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " Sep 19 09:15:20.905799 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905581 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsqf2\" (UniqueName: \"kubernetes.io/projected/495c6114-4f4b-482c-b409-995b97f62002-kube-api-access-rsqf2\") pod \"495c6114-4f4b-482c-b409-995b97f62002\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " Sep 19 09:15:20.905799 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905663 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-console-config\") pod \"495c6114-4f4b-482c-b409-995b97f62002\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " Sep 19 09:15:20.905799 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905711 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-oauth-serving-cert\") pod \"495c6114-4f4b-482c-b409-995b97f62002\" (UID: \"495c6114-4f4b-482c-b409-995b97f62002\") " Sep 19 09:15:20.905985 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905913 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-service-ca" (OuterVolumeSpecName: "service-ca") pod "495c6114-4f4b-482c-b409-995b97f62002" (UID: "495c6114-4f4b-482c-b409-995b97f62002"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:15:20.905985 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.905935 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "495c6114-4f4b-482c-b409-995b97f62002" (UID: "495c6114-4f4b-482c-b409-995b97f62002"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:15:20.906156 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.906128 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-console-config" (OuterVolumeSpecName: "console-config") pod "495c6114-4f4b-482c-b409-995b97f62002" (UID: "495c6114-4f4b-482c-b409-995b97f62002"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:15:20.906265 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.906222 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "495c6114-4f4b-482c-b409-995b97f62002" (UID: "495c6114-4f4b-482c-b409-995b97f62002"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:15:20.907918 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.907885 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/495c6114-4f4b-482c-b409-995b97f62002-kube-api-access-rsqf2" (OuterVolumeSpecName: "kube-api-access-rsqf2") pod "495c6114-4f4b-482c-b409-995b97f62002" (UID: "495c6114-4f4b-482c-b409-995b97f62002"). InnerVolumeSpecName "kube-api-access-rsqf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:15:20.908127 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.908104 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "495c6114-4f4b-482c-b409-995b97f62002" (UID: "495c6114-4f4b-482c-b409-995b97f62002"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:15:20.908127 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:20.908118 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "495c6114-4f4b-482c-b409-995b97f62002" (UID: "495c6114-4f4b-482c-b409-995b97f62002"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:15:21.006964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.006933 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:15:21.006964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.006957 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsqf2\" (UniqueName: \"kubernetes.io/projected/495c6114-4f4b-482c-b409-995b97f62002-kube-api-access-rsqf2\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:15:21.006964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.006966 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-console-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:15:21.006964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.006975 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-oauth-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:15:21.007206 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.006983 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/495c6114-4f4b-482c-b409-995b97f62002-console-oauth-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:15:21.007206 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.006991 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-trusted-ca-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:15:21.007206 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.006998 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/495c6114-4f4b-482c-b409-995b97f62002-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:15:21.605977 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.605943 2585 generic.go:356] "Generic (PLEG): container finished" podID="495c6114-4f4b-482c-b409-995b97f62002" containerID="ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f" exitCode=0 Sep 19 09:15:21.606377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.605989 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f4986c9b6-gxjbk" Sep 19 09:15:21.606377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.606018 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4986c9b6-gxjbk" event={"ID":"495c6114-4f4b-482c-b409-995b97f62002","Type":"ContainerDied","Data":"ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f"} Sep 19 09:15:21.606377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.606055 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f4986c9b6-gxjbk" event={"ID":"495c6114-4f4b-482c-b409-995b97f62002","Type":"ContainerDied","Data":"cad30bed60a14ef62caa97cca916e97a52356e494635cee0d391a58901d8106e"} Sep 19 09:15:21.606377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.606073 2585 scope.go:117] "RemoveContainer" containerID="ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f" Sep 19 09:15:21.614309 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.614292 2585 scope.go:117] "RemoveContainer" containerID="ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f" Sep 19 09:15:21.614573 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:15:21.614554 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f\": container with ID starting with ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f not found: ID does not exist" containerID="ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f" Sep 19 09:15:21.614626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.614580 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f"} err="failed to get container status \"ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f\": rpc error: code = NotFound desc = could not find container \"ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f\": container with ID starting with ede35fd174dd65a433d968ac256d59cf7ed017106098efca8af4f030ac67ef4f not found: ID does not exist" Sep 19 09:15:21.623465 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.623444 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f4986c9b6-gxjbk"] Sep 19 09:15:21.628006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:21.627983 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f4986c9b6-gxjbk"] Sep 19 09:15:23.435174 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:15:23.435142 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="495c6114-4f4b-482c-b409-995b97f62002" path="/var/lib/kubelet/pods/495c6114-4f4b-482c-b409-995b97f62002/volumes" Sep 19 09:16:35.400674 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:16:35.400644 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:16:35.401215 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:16:35.401131 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:16:35.403993 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:16:35.403974 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:16:35.404334 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:16:35.404317 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:17:10.294073 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.293997 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9"] Sep 19 09:17:10.294500 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.294266 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="495c6114-4f4b-482c-b409-995b97f62002" containerName="console" Sep 19 09:17:10.294500 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.294278 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="495c6114-4f4b-482c-b409-995b97f62002" containerName="console" Sep 19 09:17:10.294500 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.294324 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="495c6114-4f4b-482c-b409-995b97f62002" containerName="console" Sep 19 09:17:10.297330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.297314 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.300002 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.299981 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 19 09:17:10.300126 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.300049 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 19 09:17:10.301032 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.301014 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kklnq\"" Sep 19 09:17:10.318743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.318719 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9"] Sep 19 09:17:10.370500 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.370470 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.370661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.370506 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j8ls\" (UniqueName: \"kubernetes.io/projected/3aa02ac6-de0f-485a-bcf8-125fc128edc1-kube-api-access-7j8ls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.370661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.370548 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.471118 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.471089 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.471286 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.471130 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j8ls\" (UniqueName: \"kubernetes.io/projected/3aa02ac6-de0f-485a-bcf8-125fc128edc1-kube-api-access-7j8ls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.471286 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.471154 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.471587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.471567 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.471635 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.471577 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.481665 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.481635 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j8ls\" (UniqueName: \"kubernetes.io/projected/3aa02ac6-de0f-485a-bcf8-125fc128edc1-kube-api-access-7j8ls\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.605692 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.605646 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:10.875006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.874982 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9"] Sep 19 09:17:10.877467 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:17:10.877441 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa02ac6_de0f_485a_bcf8_125fc128edc1.slice/crio-618bb8ffde90ee28f56c4111d060ef2ac9caa2f27b140d461b214287b889bf99 WatchSource:0}: Error finding container 618bb8ffde90ee28f56c4111d060ef2ac9caa2f27b140d461b214287b889bf99: Status 404 returned error can't find the container with id 618bb8ffde90ee28f56c4111d060ef2ac9caa2f27b140d461b214287b889bf99 Sep 19 09:17:10.879475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.879459 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:17:10.991352 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:10.991322 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" event={"ID":"3aa02ac6-de0f-485a-bcf8-125fc128edc1","Type":"ContainerStarted","Data":"618bb8ffde90ee28f56c4111d060ef2ac9caa2f27b140d461b214287b889bf99"} Sep 19 09:17:18.020270 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:18.020232 2585 generic.go:356] "Generic (PLEG): container finished" podID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerID="93c55ec2cfe867b9e3c02eaf601087293999d551519ffbeaedfb60df134f5df5" exitCode=0 Sep 19 09:17:18.020765 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:18.020295 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" event={"ID":"3aa02ac6-de0f-485a-bcf8-125fc128edc1","Type":"ContainerDied","Data":"93c55ec2cfe867b9e3c02eaf601087293999d551519ffbeaedfb60df134f5df5"} Sep 19 09:17:21.034191 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:21.034151 2585 generic.go:356] "Generic (PLEG): container finished" podID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerID="654a8450347084877927c1ce77beffd5626dcf842afd2fbc052c5e1ff307251e" exitCode=0 Sep 19 09:17:21.034674 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:21.034238 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" event={"ID":"3aa02ac6-de0f-485a-bcf8-125fc128edc1","Type":"ContainerDied","Data":"654a8450347084877927c1ce77beffd5626dcf842afd2fbc052c5e1ff307251e"} Sep 19 09:17:28.065928 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:28.065891 2585 generic.go:356] "Generic (PLEG): container finished" podID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerID="583af0af9275f3558182f1a61a2336e9f3e9a7def646b9225fadc7e09bf6fe9e" exitCode=0 Sep 19 09:17:28.066291 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:28.065976 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" event={"ID":"3aa02ac6-de0f-485a-bcf8-125fc128edc1","Type":"ContainerDied","Data":"583af0af9275f3558182f1a61a2336e9f3e9a7def646b9225fadc7e09bf6fe9e"} Sep 19 09:17:29.183404 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.183384 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:29.331959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.331868 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-bundle\") pod \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " Sep 19 09:17:29.331959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.331927 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j8ls\" (UniqueName: \"kubernetes.io/projected/3aa02ac6-de0f-485a-bcf8-125fc128edc1-kube-api-access-7j8ls\") pod \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " Sep 19 09:17:29.331959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.331962 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-util\") pod \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\" (UID: \"3aa02ac6-de0f-485a-bcf8-125fc128edc1\") " Sep 19 09:17:29.332547 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.332496 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-bundle" (OuterVolumeSpecName: "bundle") pod "3aa02ac6-de0f-485a-bcf8-125fc128edc1" (UID: "3aa02ac6-de0f-485a-bcf8-125fc128edc1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:17:29.334143 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.334115 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aa02ac6-de0f-485a-bcf8-125fc128edc1-kube-api-access-7j8ls" (OuterVolumeSpecName: "kube-api-access-7j8ls") pod "3aa02ac6-de0f-485a-bcf8-125fc128edc1" (UID: "3aa02ac6-de0f-485a-bcf8-125fc128edc1"). InnerVolumeSpecName "kube-api-access-7j8ls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:17:29.335985 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.335961 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-util" (OuterVolumeSpecName: "util") pod "3aa02ac6-de0f-485a-bcf8-125fc128edc1" (UID: "3aa02ac6-de0f-485a-bcf8-125fc128edc1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:17:29.432513 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.432481 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:17:29.432513 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.432514 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j8ls\" (UniqueName: \"kubernetes.io/projected/3aa02ac6-de0f-485a-bcf8-125fc128edc1-kube-api-access-7j8ls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:17:29.432705 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:29.432548 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aa02ac6-de0f-485a-bcf8-125fc128edc1-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:17:30.075511 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:30.075479 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" event={"ID":"3aa02ac6-de0f-485a-bcf8-125fc128edc1","Type":"ContainerDied","Data":"618bb8ffde90ee28f56c4111d060ef2ac9caa2f27b140d461b214287b889bf99"} Sep 19 09:17:30.075511 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:30.075495 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540khjb9" Sep 19 09:17:30.075718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:30.075536 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="618bb8ffde90ee28f56c4111d060ef2ac9caa2f27b140d461b214287b889bf99" Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486227 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s"] Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486789 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerName="util" Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486807 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerName="util" Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486832 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerName="extract" Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486840 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerName="extract" Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486864 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerName="pull" Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486872 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerName="pull" Sep 19 09:17:33.487746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.486980 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa02ac6-de0f-485a-bcf8-125fc128edc1" containerName="extract" Sep 19 09:17:33.533024 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.532994 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s"] Sep 19 09:17:33.533171 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.533082 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:33.535647 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.535624 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:17:33.535768 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.535749 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-vqlbq\"" Sep 19 09:17:33.535867 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.535852 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Sep 19 09:17:33.665793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.665757 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvztd\" (UniqueName: \"kubernetes.io/projected/97e51ea9-2d44-456a-a5a4-26db3aa56b1c-kube-api-access-pvztd\") pod \"cert-manager-operator-controller-manager-55c444bf86-4c72s\" (UID: \"97e51ea9-2d44-456a-a5a4-26db3aa56b1c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:33.665935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.665827 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e51ea9-2d44-456a-a5a4-26db3aa56b1c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4c72s\" (UID: \"97e51ea9-2d44-456a-a5a4-26db3aa56b1c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:33.766660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.766560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvztd\" (UniqueName: \"kubernetes.io/projected/97e51ea9-2d44-456a-a5a4-26db3aa56b1c-kube-api-access-pvztd\") pod \"cert-manager-operator-controller-manager-55c444bf86-4c72s\" (UID: \"97e51ea9-2d44-456a-a5a4-26db3aa56b1c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:33.766660 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.766633 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e51ea9-2d44-456a-a5a4-26db3aa56b1c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4c72s\" (UID: \"97e51ea9-2d44-456a-a5a4-26db3aa56b1c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:33.767010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.766990 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/97e51ea9-2d44-456a-a5a4-26db3aa56b1c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4c72s\" (UID: \"97e51ea9-2d44-456a-a5a4-26db3aa56b1c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:33.778821 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.778799 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvztd\" (UniqueName: \"kubernetes.io/projected/97e51ea9-2d44-456a-a5a4-26db3aa56b1c-kube-api-access-pvztd\") pod \"cert-manager-operator-controller-manager-55c444bf86-4c72s\" (UID: \"97e51ea9-2d44-456a-a5a4-26db3aa56b1c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:33.841910 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:33.841878 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" Sep 19 09:17:34.131622 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:34.131599 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s"] Sep 19 09:17:34.133534 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:17:34.133496 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e51ea9_2d44_456a_a5a4_26db3aa56b1c.slice/crio-1ca83702e000578c9bc082cb750026b616570b6982464c5a889a901b694bb174 WatchSource:0}: Error finding container 1ca83702e000578c9bc082cb750026b616570b6982464c5a889a901b694bb174: Status 404 returned error can't find the container with id 1ca83702e000578c9bc082cb750026b616570b6982464c5a889a901b694bb174 Sep 19 09:17:35.096586 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:35.096551 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" event={"ID":"97e51ea9-2d44-456a-a5a4-26db3aa56b1c","Type":"ContainerStarted","Data":"1ca83702e000578c9bc082cb750026b616570b6982464c5a889a901b694bb174"} Sep 19 09:17:37.107559 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:37.107512 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" event={"ID":"97e51ea9-2d44-456a-a5a4-26db3aa56b1c","Type":"ContainerStarted","Data":"2e9734c69d6190e33ac3f4ba1464eaa43703ca422ce0b89949edd715c9526df1"} Sep 19 09:17:37.130766 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:17:37.130720 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4c72s" podStartSLOduration=2.104849503 podStartE2EDuration="4.130704743s" totalImagesPullingTime="2.02585524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:17:37.129533107 +0000 UTC m=+663.887654123" watchObservedRunningTime="2026-09-19 09:17:37.130704743 +0000 UTC m=+663.888825747" Sep 19 09:18:02.054648 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.054615 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz"] Sep 19 09:18:02.056867 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.056846 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.059135 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.059113 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 19 09:18:02.059894 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.059877 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:18:02.059992 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.059910 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ph8xs\"" Sep 19 09:18:02.067058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.067036 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz"] Sep 19 09:18:02.080990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.080962 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dtx6\" (UniqueName: \"kubernetes.io/projected/ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d-kube-api-access-2dtx6\") pod \"openshift-lws-operator-fd8ccff4c-zm5zz\" (UID: \"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.081099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.081004 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zm5zz\" (UID: \"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.181697 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.181662 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zm5zz\" (UID: \"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.181861 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.181747 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dtx6\" (UniqueName: \"kubernetes.io/projected/ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d-kube-api-access-2dtx6\") pod \"openshift-lws-operator-fd8ccff4c-zm5zz\" (UID: \"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.182060 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.182040 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zm5zz\" (UID: \"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.190958 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.190928 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dtx6\" (UniqueName: \"kubernetes.io/projected/ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d-kube-api-access-2dtx6\") pod \"openshift-lws-operator-fd8ccff4c-zm5zz\" (UID: \"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.365558 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.365485 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" Sep 19 09:18:02.633803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:02.633778 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz"] Sep 19 09:18:02.636290 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:18:02.636262 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac0fba72_bc18_4119_b8ab_8ad6d7c62b9d.slice/crio-13f95280b6e7d5268af21b26c22aa39c52e0dc70a29448809c13ee7d450e8b17 WatchSource:0}: Error finding container 13f95280b6e7d5268af21b26c22aa39c52e0dc70a29448809c13ee7d450e8b17: Status 404 returned error can't find the container with id 13f95280b6e7d5268af21b26c22aa39c52e0dc70a29448809c13ee7d450e8b17 Sep 19 09:18:03.216591 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:03.216557 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" event={"ID":"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d","Type":"ContainerStarted","Data":"13f95280b6e7d5268af21b26c22aa39c52e0dc70a29448809c13ee7d450e8b17"} Sep 19 09:18:06.232877 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:06.232842 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" event={"ID":"ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d","Type":"ContainerStarted","Data":"294c91d4ecfde50e5f8a45b43cd26b9a48e0d74c3796cda1548405ce5b3693bc"} Sep 19 09:18:06.249654 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:06.249610 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zm5zz" podStartSLOduration=1.602614314 podStartE2EDuration="4.249593646s" totalImagesPullingTime="2.646979332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:06.248873852 +0000 UTC m=+693.006994876" watchObservedRunningTime="2026-09-19 09:18:06.249593646 +0000 UTC m=+693.007714670" Sep 19 09:18:11.171403 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.171370 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dc5b6d78b-6fl6p"] Sep 19 09:18:11.173568 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.173513 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.191765 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.191742 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc5b6d78b-6fl6p"] Sep 19 09:18:11.244255 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.244230 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcl74\" (UniqueName: \"kubernetes.io/projected/068ce7f8-360f-483d-a256-77eeca4f631c-kube-api-access-pcl74\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.244366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.244266 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-console-config\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.244366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.244287 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-service-ca\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.244366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.244323 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068ce7f8-360f-483d-a256-77eeca4f631c-console-serving-cert\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.244366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.244350 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068ce7f8-360f-483d-a256-77eeca4f631c-console-oauth-config\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.244497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.244387 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-oauth-serving-cert\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.244497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.244405 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-trusted-ca-bundle\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.345615 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.345588 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-console-config\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.345769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.345625 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-service-ca\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.345769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.345645 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068ce7f8-360f-483d-a256-77eeca4f631c-console-serving-cert\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.345769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.345667 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068ce7f8-360f-483d-a256-77eeca4f631c-console-oauth-config\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.345769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.345713 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-oauth-serving-cert\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.345769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.345734 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-trusted-ca-bundle\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.345769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.345766 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcl74\" (UniqueName: \"kubernetes.io/projected/068ce7f8-360f-483d-a256-77eeca4f631c-kube-api-access-pcl74\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.346369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.346344 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-console-config\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.346487 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.346449 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-oauth-serving-cert\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.346566 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.346552 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-service-ca\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.346711 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.346687 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/068ce7f8-360f-483d-a256-77eeca4f631c-trusted-ca-bundle\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.348115 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.348093 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/068ce7f8-360f-483d-a256-77eeca4f631c-console-serving-cert\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.348226 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.348212 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/068ce7f8-360f-483d-a256-77eeca4f631c-console-oauth-config\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.354122 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.354105 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcl74\" (UniqueName: \"kubernetes.io/projected/068ce7f8-360f-483d-a256-77eeca4f631c-kube-api-access-pcl74\") pod \"console-6dc5b6d78b-6fl6p\" (UID: \"068ce7f8-360f-483d-a256-77eeca4f631c\") " pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.483586 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.483515 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:11.750622 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:11.750598 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc5b6d78b-6fl6p"] Sep 19 09:18:11.752848 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:18:11.752822 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068ce7f8_360f_483d_a256_77eeca4f631c.slice/crio-9141f92985dec4b1a6f957576da0003b5f8b25f513f0b215e333516cdbb060f2 WatchSource:0}: Error finding container 9141f92985dec4b1a6f957576da0003b5f8b25f513f0b215e333516cdbb060f2: Status 404 returned error can't find the container with id 9141f92985dec4b1a6f957576da0003b5f8b25f513f0b215e333516cdbb060f2 Sep 19 09:18:12.263595 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:12.263553 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc5b6d78b-6fl6p" event={"ID":"068ce7f8-360f-483d-a256-77eeca4f631c","Type":"ContainerStarted","Data":"04bf1beb5a339922a0d3e6612a05262db96766a651e44a1da9908cf5b5e4f310"} Sep 19 09:18:12.264043 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:12.263609 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc5b6d78b-6fl6p" event={"ID":"068ce7f8-360f-483d-a256-77eeca4f631c","Type":"ContainerStarted","Data":"9141f92985dec4b1a6f957576da0003b5f8b25f513f0b215e333516cdbb060f2"} Sep 19 09:18:12.281654 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:12.281608 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dc5b6d78b-6fl6p" podStartSLOduration=1.281592206 podStartE2EDuration="1.281592206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:12.280703334 +0000 UTC m=+699.038824361" watchObservedRunningTime="2026-09-19 09:18:12.281592206 +0000 UTC m=+699.039713230" Sep 19 09:18:21.407750 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.407717 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62"] Sep 19 09:18:21.410914 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.410897 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.413427 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.413406 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 19 09:18:21.413547 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.413492 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 19 09:18:21.414217 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.414195 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 19 09:18:21.414912 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.414893 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 19 09:18:21.414989 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.414894 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7pvgm\"" Sep 19 09:18:21.436154 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.436136 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62"] Sep 19 09:18:21.484044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.484019 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:21.484177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.484067 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:21.488323 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.488303 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:21.521177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.521151 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9dfa6654-1703-433e-a300-07cfa060c67d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.521273 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.521181 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9dfa6654-1703-433e-a300-07cfa060c67d-manifests\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.521273 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.521224 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dfa6654-1703-433e-a300-07cfa060c67d-tmp\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.521273 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.521242 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9dfa6654-1703-433e-a300-07cfa060c67d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.521408 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.521311 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzbqv\" (UniqueName: \"kubernetes.io/projected/9dfa6654-1703-433e-a300-07cfa060c67d-kube-api-access-lzbqv\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.621998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.621973 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9dfa6654-1703-433e-a300-07cfa060c67d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.622092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.621999 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9dfa6654-1703-433e-a300-07cfa060c67d-manifests\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.622092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.622018 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dfa6654-1703-433e-a300-07cfa060c67d-tmp\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.622092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.622034 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9dfa6654-1703-433e-a300-07cfa060c67d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.622092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.622078 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzbqv\" (UniqueName: \"kubernetes.io/projected/9dfa6654-1703-433e-a300-07cfa060c67d-kube-api-access-lzbqv\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.622338 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.622320 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9dfa6654-1703-433e-a300-07cfa060c67d-manifests\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.624184 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.624164 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dfa6654-1703-433e-a300-07cfa060c67d-tmp\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.624428 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.624414 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9dfa6654-1703-433e-a300-07cfa060c67d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.624428 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.624420 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9dfa6654-1703-433e-a300-07cfa060c67d-webhook-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.648287 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.648270 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzbqv\" (UniqueName: \"kubernetes.io/projected/9dfa6654-1703-433e-a300-07cfa060c67d-kube-api-access-lzbqv\") pod \"opendatahub-operator-controller-manager-56479975c9-pkb62\" (UID: \"9dfa6654-1703-433e-a300-07cfa060c67d\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:21.721439 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:21.721392 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:22.002111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:22.002038 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62"] Sep 19 09:18:22.005358 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:18:22.005329 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dfa6654_1703_433e_a300_07cfa060c67d.slice/crio-4c1ab9ae3237656050034b9d5d08641eac36f80cbb63de924126556a2fcd77c8 WatchSource:0}: Error finding container 4c1ab9ae3237656050034b9d5d08641eac36f80cbb63de924126556a2fcd77c8: Status 404 returned error can't find the container with id 4c1ab9ae3237656050034b9d5d08641eac36f80cbb63de924126556a2fcd77c8 Sep 19 09:18:22.306374 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:22.306341 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" event={"ID":"9dfa6654-1703-433e-a300-07cfa060c67d","Type":"ContainerStarted","Data":"4c1ab9ae3237656050034b9d5d08641eac36f80cbb63de924126556a2fcd77c8"} Sep 19 09:18:22.310126 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:22.310103 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dc5b6d78b-6fl6p" Sep 19 09:18:22.362721 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:22.362692 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9f5846bc5-7t25z"] Sep 19 09:18:25.321956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:25.321913 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" event={"ID":"9dfa6654-1703-433e-a300-07cfa060c67d","Type":"ContainerStarted","Data":"e941e168e4dc98acab5a24412c4a5684d3ffa63c7b22c3f6b4811e174f874891"} Sep 19 09:18:26.328101 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:26.328063 2585 generic.go:356] "Generic (PLEG): container finished" podID="9dfa6654-1703-433e-a300-07cfa060c67d" containerID="e941e168e4dc98acab5a24412c4a5684d3ffa63c7b22c3f6b4811e174f874891" exitCode=0 Sep 19 09:18:26.328492 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:26.328149 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" event={"ID":"9dfa6654-1703-433e-a300-07cfa060c67d","Type":"ContainerDied","Data":"e941e168e4dc98acab5a24412c4a5684d3ffa63c7b22c3f6b4811e174f874891"} Sep 19 09:18:27.335390 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:27.335357 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" event={"ID":"9dfa6654-1703-433e-a300-07cfa060c67d","Type":"ContainerStarted","Data":"4b77f069ebe693ed65679d16c344d42c1834e1bcae60872f0b8acb9482845123"} Sep 19 09:18:27.335875 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:27.335473 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:27.387420 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:27.387365 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" podStartSLOduration=2.023479966 podStartE2EDuration="6.387348387s" totalImagesPullingTime="3.285045617s" totalInitContainerRuntime="1.078822804s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:27.385704749 +0000 UTC m=+714.143825785" watchObservedRunningTime="2026-09-19 09:18:27.387348387 +0000 UTC m=+714.145469475" Sep 19 09:18:38.341694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:38.341665 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-pkb62" Sep 19 09:18:41.651339 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.651297 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht"] Sep 19 09:18:41.660416 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.660391 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.668971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.668945 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 19 09:18:41.669143 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.669117 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pwbkc\"" Sep 19 09:18:41.669738 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.669722 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 19 09:18:41.679733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.679709 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht"] Sep 19 09:18:41.794782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.794746 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zfj5\" (UniqueName: \"kubernetes.io/projected/9688666a-3a15-4cc9-b226-4ab172e41fea-kube-api-access-4zfj5\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.794963 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.794822 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9688666a-3a15-4cc9-b226-4ab172e41fea-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.794963 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.794922 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9688666a-3a15-4cc9-b226-4ab172e41fea-tmp\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.895870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.895833 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9688666a-3a15-4cc9-b226-4ab172e41fea-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.896067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.895884 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9688666a-3a15-4cc9-b226-4ab172e41fea-tmp\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.896067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.895929 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfj5\" (UniqueName: \"kubernetes.io/projected/9688666a-3a15-4cc9-b226-4ab172e41fea-kube-api-access-4zfj5\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.896067 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:18:41.895972 2585 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 19 09:18:41.896067 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:18:41.896045 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9688666a-3a15-4cc9-b226-4ab172e41fea-tls-certs podName:9688666a-3a15-4cc9-b226-4ab172e41fea nodeName:}" failed. No retries permitted until 2026-09-19 09:18:42.396028105 +0000 UTC m=+729.154149146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9688666a-3a15-4cc9-b226-4ab172e41fea-tls-certs") pod "kube-auth-proxy-7d645f99d7-gj9ht" (UID: "9688666a-3a15-4cc9-b226-4ab172e41fea") : secret "kube-auth-proxy-tls" not found Sep 19 09:18:41.898178 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.898152 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9688666a-3a15-4cc9-b226-4ab172e41fea-tmp\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:41.914974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:41.914891 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfj5\" (UniqueName: \"kubernetes.io/projected/9688666a-3a15-4cc9-b226-4ab172e41fea-kube-api-access-4zfj5\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:42.402028 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:42.401996 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9688666a-3a15-4cc9-b226-4ab172e41fea-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:42.404396 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:42.404365 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9688666a-3a15-4cc9-b226-4ab172e41fea-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-gj9ht\" (UID: \"9688666a-3a15-4cc9-b226-4ab172e41fea\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:42.570203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:42.570162 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" Sep 19 09:18:42.866077 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:42.866041 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht"] Sep 19 09:18:42.868908 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:18:42.868877 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9688666a_3a15_4cc9_b226_4ab172e41fea.slice/crio-f3addf2adf07e05e70ef61fcbe8864487d321c4c6fe8746a8d09c4a504d12b11 WatchSource:0}: Error finding container f3addf2adf07e05e70ef61fcbe8864487d321c4c6fe8746a8d09c4a504d12b11: Status 404 returned error can't find the container with id f3addf2adf07e05e70ef61fcbe8864487d321c4c6fe8746a8d09c4a504d12b11 Sep 19 09:18:43.344218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.344185 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8"] Sep 19 09:18:43.348360 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.348340 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.350682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.350651 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 19 09:18:43.350682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.350656 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 19 09:18:43.351470 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.351456 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 19 09:18:43.351839 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.351821 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pvvx2\"" Sep 19 09:18:43.352369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.352348 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 19 09:18:43.364872 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.364845 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8"] Sep 19 09:18:43.405763 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.405727 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" event={"ID":"9688666a-3a15-4cc9-b226-4ab172e41fea","Type":"ContainerStarted","Data":"f3addf2adf07e05e70ef61fcbe8864487d321c4c6fe8746a8d09c4a504d12b11"} Sep 19 09:18:43.512292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.512259 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl5rx\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-kube-api-access-vl5rx\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.512484 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.512323 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.512484 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.512402 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.512484 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.512444 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.512484 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.512480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35260a2a-80fe-4c0a-8a30-5eb88cff036f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.512733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.512540 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.512733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.512561 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.613773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.613672 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl5rx\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-kube-api-access-vl5rx\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.613773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.613754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.613996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.613795 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.613996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.613825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.613996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.613860 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35260a2a-80fe-4c0a-8a30-5eb88cff036f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.613996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.613917 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.613996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.613944 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.614786 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.614757 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.617333 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.617302 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.617558 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.617504 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35260a2a-80fe-4c0a-8a30-5eb88cff036f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.617723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.617575 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.618200 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.618182 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.623064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.622987 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl5rx\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-kube-api-access-vl5rx\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.623182 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.623094 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bljm8\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.659052 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.659025 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:43.948843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:43.948746 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8"] Sep 19 09:18:43.952597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:18:43.952566 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35260a2a_80fe_4c0a_8a30_5eb88cff036f.slice/crio-ddc84d7a86b477b15680c5c16e9942b1169759cb123e70ae009df3413a5645e5 WatchSource:0}: Error finding container ddc84d7a86b477b15680c5c16e9942b1169759cb123e70ae009df3413a5645e5: Status 404 returned error can't find the container with id ddc84d7a86b477b15680c5c16e9942b1169759cb123e70ae009df3413a5645e5 Sep 19 09:18:44.413732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:44.413691 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" event={"ID":"35260a2a-80fe-4c0a-8a30-5eb88cff036f","Type":"ContainerStarted","Data":"ddc84d7a86b477b15680c5c16e9942b1169759cb123e70ae009df3413a5645e5"} Sep 19 09:18:46.426030 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:46.425988 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" event={"ID":"9688666a-3a15-4cc9-b226-4ab172e41fea","Type":"ContainerStarted","Data":"b4bd30041ff6038ff0519dbbf610e101c01651b416131e1d65c5e32b8782f004"} Sep 19 09:18:46.444411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:46.444356 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-gj9ht" podStartSLOduration=2.583994513 podStartE2EDuration="5.444339827s" totalImagesPullingTime="2.860345314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:46.442205952 +0000 UTC m=+733.200326977" watchObservedRunningTime="2026-09-19 09:18:46.444339827 +0000 UTC m=+733.202460851" Sep 19 09:18:47.384404 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.384357 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9f5846bc5-7t25z" podUID="c3dc326f-0d53-4bb7-868a-db343e04a118" containerName="console" containerID="cri-o://6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f" gracePeriod=15 Sep 19 09:18:47.462084 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.462050 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:18:47.462354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.462123 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:18:47.625916 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.625887 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:18:47.760244 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760171 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-serving-cert\") pod \"c3dc326f-0d53-4bb7-868a-db343e04a118\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " Sep 19 09:18:47.760244 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760203 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-oauth-serving-cert\") pod \"c3dc326f-0d53-4bb7-868a-db343e04a118\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " Sep 19 09:18:47.760448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760254 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-service-ca\") pod \"c3dc326f-0d53-4bb7-868a-db343e04a118\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " Sep 19 09:18:47.760448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760271 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-oauth-config\") pod \"c3dc326f-0d53-4bb7-868a-db343e04a118\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " Sep 19 09:18:47.760448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760290 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-console-config\") pod \"c3dc326f-0d53-4bb7-868a-db343e04a118\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " Sep 19 09:18:47.760448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760313 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lktfk\" (UniqueName: \"kubernetes.io/projected/c3dc326f-0d53-4bb7-868a-db343e04a118-kube-api-access-lktfk\") pod \"c3dc326f-0d53-4bb7-868a-db343e04a118\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " Sep 19 09:18:47.760672 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760624 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-trusted-ca-bundle\") pod \"c3dc326f-0d53-4bb7-868a-db343e04a118\" (UID: \"c3dc326f-0d53-4bb7-868a-db343e04a118\") " Sep 19 09:18:47.760758 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760731 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-console-config" (OuterVolumeSpecName: "console-config") pod "c3dc326f-0d53-4bb7-868a-db343e04a118" (UID: "c3dc326f-0d53-4bb7-868a-db343e04a118"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:18:47.760758 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760745 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c3dc326f-0d53-4bb7-868a-db343e04a118" (UID: "c3dc326f-0d53-4bb7-868a-db343e04a118"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:18:47.760881 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760752 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-service-ca" (OuterVolumeSpecName: "service-ca") pod "c3dc326f-0d53-4bb7-868a-db343e04a118" (UID: "c3dc326f-0d53-4bb7-868a-db343e04a118"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:18:47.760935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760904 2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:18:47.760935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760922 2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-console-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:18:47.761032 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.760935 2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-oauth-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:18:47.761119 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.761089 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c3dc326f-0d53-4bb7-868a-db343e04a118" (UID: "c3dc326f-0d53-4bb7-868a-db343e04a118"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:18:47.762548 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.762504 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c3dc326f-0d53-4bb7-868a-db343e04a118" (UID: "c3dc326f-0d53-4bb7-868a-db343e04a118"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:18:47.762971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.762949 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c3dc326f-0d53-4bb7-868a-db343e04a118" (UID: "c3dc326f-0d53-4bb7-868a-db343e04a118"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:18:47.763072 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.762971 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3dc326f-0d53-4bb7-868a-db343e04a118-kube-api-access-lktfk" (OuterVolumeSpecName: "kube-api-access-lktfk") pod "c3dc326f-0d53-4bb7-868a-db343e04a118" (UID: "c3dc326f-0d53-4bb7-868a-db343e04a118"). InnerVolumeSpecName "kube-api-access-lktfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:18:47.862194 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.862100 2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dc326f-0d53-4bb7-868a-db343e04a118-trusted-ca-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:18:47.862194 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.862138 2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-serving-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:18:47.862194 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.862152 2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3dc326f-0d53-4bb7-868a-db343e04a118-console-oauth-config\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:18:47.862194 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:47.862164 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lktfk\" (UniqueName: \"kubernetes.io/projected/c3dc326f-0d53-4bb7-868a-db343e04a118-kube-api-access-lktfk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:18:48.438003 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.437967 2585 generic.go:356] "Generic (PLEG): container finished" podID="c3dc326f-0d53-4bb7-868a-db343e04a118" containerID="6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f" exitCode=0 Sep 19 09:18:48.438191 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.438008 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5846bc5-7t25z" Sep 19 09:18:48.438191 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.438042 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5846bc5-7t25z" event={"ID":"c3dc326f-0d53-4bb7-868a-db343e04a118","Type":"ContainerDied","Data":"6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f"} Sep 19 09:18:48.438191 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.438084 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5846bc5-7t25z" event={"ID":"c3dc326f-0d53-4bb7-868a-db343e04a118","Type":"ContainerDied","Data":"6b105c5dbbc10bfce43d6fc2bff57cd964f1239e81a765f3d6e080a63d1a9f50"} Sep 19 09:18:48.438191 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.438107 2585 scope.go:117] "RemoveContainer" containerID="6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f" Sep 19 09:18:48.439822 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.439801 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" event={"ID":"35260a2a-80fe-4c0a-8a30-5eb88cff036f","Type":"ContainerStarted","Data":"a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd"} Sep 19 09:18:48.440040 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.440017 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:18:48.441976 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.441956 2585 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-bljm8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 19 09:18:48.442074 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.442013 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" podUID="35260a2a-80fe-4c0a-8a30-5eb88cff036f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 09:18:48.448011 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.447995 2585 scope.go:117] "RemoveContainer" containerID="6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f" Sep 19 09:18:48.448262 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:18:48.448245 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f\": container with ID starting with 6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f not found: ID does not exist" containerID="6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f" Sep 19 09:18:48.448315 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.448270 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f"} err="failed to get container status \"6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f\": rpc error: code = NotFound desc = could not find container \"6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f\": container with ID starting with 6866ae4bf3670d8036d5d6ffd51a7a9d9caaa13db2d1a85c637c0a28afb35e9f not found: ID does not exist" Sep 19 09:18:48.460436 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.460395 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" podStartSLOduration=1.9538422340000001 podStartE2EDuration="5.4603826s" totalImagesPullingTime="3.506540366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:48.458633623 +0000 UTC m=+735.216754648" watchObservedRunningTime="2026-09-19 09:18:48.4603826 +0000 UTC m=+735.218503625" Sep 19 09:18:48.472679 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.472659 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9f5846bc5-7t25z"] Sep 19 09:18:48.477065 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:48.477046 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9f5846bc5-7t25z"] Sep 19 09:18:49.435022 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:49.434991 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3dc326f-0d53-4bb7-868a-db343e04a118" path="/var/lib/kubelet/pods/c3dc326f-0d53-4bb7-868a-db343e04a118/volumes" Sep 19 09:18:49.445189 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:18:49.445169 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:19:01.693911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.693873 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9sqht"] Sep 19 09:19:01.694499 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.694273 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3dc326f-0d53-4bb7-868a-db343e04a118" containerName="console" Sep 19 09:19:01.694499 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.694289 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3dc326f-0d53-4bb7-868a-db343e04a118" containerName="console" Sep 19 09:19:01.694499 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.694353 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3dc326f-0d53-4bb7-868a-db343e04a118" containerName="console" Sep 19 09:19:01.704952 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.704930 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:01.708138 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.708101 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z"] Sep 19 09:19:01.708253 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.708187 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 19 09:19:01.708253 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.708234 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jjxv8\"" Sep 19 09:19:01.714082 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.711881 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:01.714927 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.714771 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 19 09:19:01.715916 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.715896 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9sqht"] Sep 19 09:19:01.717017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.716996 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lcn89\"" Sep 19 09:19:01.728612 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.728592 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z"] Sep 19 09:19:01.772580 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.772556 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntrh2\" (UniqueName: \"kubernetes.io/projected/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-kube-api-access-ntrh2\") pod \"kserve-controller-manager-d6f4b65b8-9sqht\" (UID: \"1336bfc7-f678-4c54-bea3-7ce8e78eda8e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:01.772718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.772585 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d2gd\" (UniqueName: \"kubernetes.io/projected/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-kube-api-access-5d2gd\") pod \"llmisvc-controller-manager-777c6bd7d5-bbk2z\" (UID: \"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:01.772718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.772606 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bbk2z\" (UID: \"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:01.772718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.772626 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-cert\") pod \"kserve-controller-manager-d6f4b65b8-9sqht\" (UID: \"1336bfc7-f678-4c54-bea3-7ce8e78eda8e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:01.873115 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.873087 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntrh2\" (UniqueName: \"kubernetes.io/projected/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-kube-api-access-ntrh2\") pod \"kserve-controller-manager-d6f4b65b8-9sqht\" (UID: \"1336bfc7-f678-4c54-bea3-7ce8e78eda8e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:01.873225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.873122 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d2gd\" (UniqueName: \"kubernetes.io/projected/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-kube-api-access-5d2gd\") pod \"llmisvc-controller-manager-777c6bd7d5-bbk2z\" (UID: \"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:01.873225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.873147 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bbk2z\" (UID: \"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:01.873225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.873181 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-cert\") pod \"kserve-controller-manager-d6f4b65b8-9sqht\" (UID: \"1336bfc7-f678-4c54-bea3-7ce8e78eda8e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:01.873348 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:19:01.873264 2585 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Sep 19 09:19:01.873348 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:19:01.873280 2585 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 19 09:19:01.873348 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:19:01.873333 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-cert podName:ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b nodeName:}" failed. No retries permitted until 2026-09-19 09:19:02.373315 +0000 UTC m=+749.131436004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-cert") pod "llmisvc-controller-manager-777c6bd7d5-bbk2z" (UID: "ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b") : secret "llmisvc-webhook-server-cert" not found Sep 19 09:19:01.873476 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:19:01.873351 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-cert podName:1336bfc7-f678-4c54-bea3-7ce8e78eda8e nodeName:}" failed. No retries permitted until 2026-09-19 09:19:02.373342417 +0000 UTC m=+749.131463424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-cert") pod "kserve-controller-manager-d6f4b65b8-9sqht" (UID: "1336bfc7-f678-4c54-bea3-7ce8e78eda8e") : secret "kserve-webhook-server-cert" not found Sep 19 09:19:01.884185 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.884156 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d2gd\" (UniqueName: \"kubernetes.io/projected/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-kube-api-access-5d2gd\") pod \"llmisvc-controller-manager-777c6bd7d5-bbk2z\" (UID: \"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:01.884632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:01.884615 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntrh2\" (UniqueName: \"kubernetes.io/projected/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-kube-api-access-ntrh2\") pod \"kserve-controller-manager-d6f4b65b8-9sqht\" (UID: \"1336bfc7-f678-4c54-bea3-7ce8e78eda8e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:02.377293 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.377261 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-cert\") pod \"kserve-controller-manager-d6f4b65b8-9sqht\" (UID: \"1336bfc7-f678-4c54-bea3-7ce8e78eda8e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:02.377464 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.377331 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bbk2z\" (UID: \"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:02.379842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.379817 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bbk2z\" (UID: \"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:02.379960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.379817 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1336bfc7-f678-4c54-bea3-7ce8e78eda8e-cert\") pod \"kserve-controller-manager-d6f4b65b8-9sqht\" (UID: \"1336bfc7-f678-4c54-bea3-7ce8e78eda8e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:02.619649 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.619618 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:02.625720 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.625700 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:02.870632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.870588 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8"] Sep 19 09:19:02.871009 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.870827 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" podUID="35260a2a-80fe-4c0a-8a30-5eb88cff036f" containerName="discovery" containerID="cri-o://a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd" gracePeriod=30 Sep 19 09:19:02.907361 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.907338 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9sqht"] Sep 19 09:19:02.914498 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:02.914477 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z"] Sep 19 09:19:02.943365 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:19:02.943338 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1336bfc7_f678_4c54_bea3_7ce8e78eda8e.slice/crio-b118285b13c78ade9e8393445e03fb855ab895e22a7166ad2f5d38fc232ad835 WatchSource:0}: Error finding container b118285b13c78ade9e8393445e03fb855ab895e22a7166ad2f5d38fc232ad835: Status 404 returned error can't find the container with id b118285b13c78ade9e8393445e03fb855ab895e22a7166ad2f5d38fc232ad835 Sep 19 09:19:02.970378 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:19:02.970346 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podca760d1f_c1ed_4bbf_8ffc_cf3b1461719b.slice/crio-41046d64e2ffd37dde1eb7a084e9fbd6162e892dc4d20255ee9e71b2500340af WatchSource:0}: Error finding container 41046d64e2ffd37dde1eb7a084e9fbd6162e892dc4d20255ee9e71b2500340af: Status 404 returned error can't find the container with id 41046d64e2ffd37dde1eb7a084e9fbd6162e892dc4d20255ee9e71b2500340af Sep 19 09:19:03.132883 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.132861 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:19:03.185229 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185162 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-ca-configmap\") pod \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " Sep 19 09:19:03.185229 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185197 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-cacerts\") pod \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " Sep 19 09:19:03.185229 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185220 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-token\") pod \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " Sep 19 09:19:03.185414 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185402 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl5rx\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-kube-api-access-vl5rx\") pod \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " Sep 19 09:19:03.185473 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185460 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-dns-cert\") pod \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " Sep 19 09:19:03.185570 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185487 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-kubeconfig\") pod \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " Sep 19 09:19:03.185570 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185505 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35260a2a-80fe-4c0a-8a30-5eb88cff036f-local-certs\") pod \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\" (UID: \"35260a2a-80fe-4c0a-8a30-5eb88cff036f\") " Sep 19 09:19:03.185810 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.185759 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "35260a2a-80fe-4c0a-8a30-5eb88cff036f" (UID: "35260a2a-80fe-4c0a-8a30-5eb88cff036f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:19:03.187718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.187677 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-cacerts" (OuterVolumeSpecName: "cacerts") pod "35260a2a-80fe-4c0a-8a30-5eb88cff036f" (UID: "35260a2a-80fe-4c0a-8a30-5eb88cff036f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:19:03.187718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.187692 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-kube-api-access-vl5rx" (OuterVolumeSpecName: "kube-api-access-vl5rx") pod "35260a2a-80fe-4c0a-8a30-5eb88cff036f" (UID: "35260a2a-80fe-4c0a-8a30-5eb88cff036f"). InnerVolumeSpecName "kube-api-access-vl5rx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:19:03.187908 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.187766 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-token" (OuterVolumeSpecName: "istio-token") pod "35260a2a-80fe-4c0a-8a30-5eb88cff036f" (UID: "35260a2a-80fe-4c0a-8a30-5eb88cff036f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:19:03.187908 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.187763 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "35260a2a-80fe-4c0a-8a30-5eb88cff036f" (UID: "35260a2a-80fe-4c0a-8a30-5eb88cff036f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:19:03.188091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.188067 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35260a2a-80fe-4c0a-8a30-5eb88cff036f-local-certs" (OuterVolumeSpecName: "local-certs") pod "35260a2a-80fe-4c0a-8a30-5eb88cff036f" (UID: "35260a2a-80fe-4c0a-8a30-5eb88cff036f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:19:03.188091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.188077 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "35260a2a-80fe-4c0a-8a30-5eb88cff036f" (UID: "35260a2a-80fe-4c0a-8a30-5eb88cff036f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:19:03.286510 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.286474 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-dns-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:19:03.286510 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.286506 2585 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-kubeconfig\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:19:03.286510 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.286515 2585 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/35260a2a-80fe-4c0a-8a30-5eb88cff036f-local-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:19:03.286743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.286544 2585 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-csr-ca-configmap\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:19:03.286743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.286553 2585 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/35260a2a-80fe-4c0a-8a30-5eb88cff036f-cacerts\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:19:03.286743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.286561 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:19:03.286743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.286568 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl5rx\" (UniqueName: \"kubernetes.io/projected/35260a2a-80fe-4c0a-8a30-5eb88cff036f-kube-api-access-vl5rx\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:19:03.514196 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.514161 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" event={"ID":"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b","Type":"ContainerStarted","Data":"41046d64e2ffd37dde1eb7a084e9fbd6162e892dc4d20255ee9e71b2500340af"} Sep 19 09:19:03.515590 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.515563 2585 generic.go:356] "Generic (PLEG): container finished" podID="35260a2a-80fe-4c0a-8a30-5eb88cff036f" containerID="a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd" exitCode=0 Sep 19 09:19:03.515732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.515614 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" Sep 19 09:19:03.515732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.515640 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" event={"ID":"35260a2a-80fe-4c0a-8a30-5eb88cff036f","Type":"ContainerDied","Data":"a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd"} Sep 19 09:19:03.515732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.515673 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8" event={"ID":"35260a2a-80fe-4c0a-8a30-5eb88cff036f","Type":"ContainerDied","Data":"ddc84d7a86b477b15680c5c16e9942b1169759cb123e70ae009df3413a5645e5"} Sep 19 09:19:03.515732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.515695 2585 scope.go:117] "RemoveContainer" containerID="a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd" Sep 19 09:19:03.517329 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.517308 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" event={"ID":"1336bfc7-f678-4c54-bea3-7ce8e78eda8e","Type":"ContainerStarted","Data":"b118285b13c78ade9e8393445e03fb855ab895e22a7166ad2f5d38fc232ad835"} Sep 19 09:19:03.535025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.534983 2585 scope.go:117] "RemoveContainer" containerID="a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd" Sep 19 09:19:03.535249 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:19:03.535227 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd\": container with ID starting with a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd not found: ID does not exist" containerID="a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd" Sep 19 09:19:03.535315 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.535256 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd"} err="failed to get container status \"a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd\": rpc error: code = NotFound desc = could not find container \"a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd\": container with ID starting with a8d1038488c60dd6b0b5e778cedc4f7407cd604170c82783e9e3aabf822b51dd not found: ID does not exist" Sep 19 09:19:03.535683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.535662 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8"] Sep 19 09:19:03.538596 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:03.538576 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bljm8"] Sep 19 09:19:05.437550 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:05.437487 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35260a2a-80fe-4c0a-8a30-5eb88cff036f" path="/var/lib/kubelet/pods/35260a2a-80fe-4c0a-8a30-5eb88cff036f/volumes" Sep 19 09:19:06.534419 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:06.534381 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" event={"ID":"1336bfc7-f678-4c54-bea3-7ce8e78eda8e","Type":"ContainerStarted","Data":"5930dbf86ab636306da8bfba98894df6f27c1e01145a239463b0d2c615c2c5c7"} Sep 19 09:19:06.534894 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:06.534447 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:06.535978 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:06.535954 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" event={"ID":"ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b","Type":"ContainerStarted","Data":"5d8b45cf80d21d6ae08e899a7e192f084a721eddf47e5d5682314a82faf72cb5"} Sep 19 09:19:06.536085 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:06.536070 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:06.575280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:06.575228 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" podStartSLOduration=2.6871681929999998 podStartE2EDuration="5.575210795s" totalImagesPullingTime="2.888042602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:19:06.574279848 +0000 UTC m=+753.332400864" watchObservedRunningTime="2026-09-19 09:19:06.575210795 +0000 UTC m=+753.333331819" Sep 19 09:19:06.612133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:06.612072 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" podStartSLOduration=2.719960872 podStartE2EDuration="5.612053124s" totalImagesPullingTime="2.892092252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:19:06.605436773 +0000 UTC m=+753.363557799" watchObservedRunningTime="2026-09-19 09:19:06.612053124 +0000 UTC m=+753.370174221" Sep 19 09:19:37.542471 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:37.542436 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bbk2z" Sep 19 09:19:37.545845 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:37.545804 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9sqht" Sep 19 09:19:57.578983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.578904 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58"] Sep 19 09:19:57.579348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.579221 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35260a2a-80fe-4c0a-8a30-5eb88cff036f" containerName="discovery" Sep 19 09:19:57.579348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.579233 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="35260a2a-80fe-4c0a-8a30-5eb88cff036f" containerName="discovery" Sep 19 09:19:57.579348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.579281 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="35260a2a-80fe-4c0a-8a30-5eb88cff036f" containerName="discovery" Sep 19 09:19:57.582785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.582764 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.585276 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.585253 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 19 09:19:57.586083 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.586063 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 19 09:19:57.586181 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.586064 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kklnq\"" Sep 19 09:19:57.591480 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.591460 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58"] Sep 19 09:19:57.621261 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.621238 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.621379 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.621289 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rz5j\" (UniqueName: \"kubernetes.io/projected/c50ad76a-468e-4b46-a981-c4c98ce88302-kube-api-access-5rz5j\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.621379 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.621335 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.675782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.675754 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf"] Sep 19 09:19:57.679380 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.679362 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.688279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.688260 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf"] Sep 19 09:19:57.722589 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.722556 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.722740 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.722606 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rz5j\" (UniqueName: \"kubernetes.io/projected/c50ad76a-468e-4b46-a981-c4c98ce88302-kube-api-access-5rz5j\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.722740 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.722674 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.722740 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.722700 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.722904 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.722740 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dp2g\" (UniqueName: \"kubernetes.io/projected/c7b29228-9cec-4a07-8fba-0dda8649d800-kube-api-access-5dp2g\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.722904 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.722785 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.723036 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.723017 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.723127 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.723108 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.732338 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.732317 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rz5j\" (UniqueName: \"kubernetes.io/projected/c50ad76a-468e-4b46-a981-c4c98ce88302-kube-api-access-5rz5j\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.774936 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.774908 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm"] Sep 19 09:19:57.778236 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.778220 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.787166 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.787140 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm"] Sep 19 09:19:57.823229 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.823376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823242 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dp2g\" (UniqueName: \"kubernetes.io/projected/c7b29228-9cec-4a07-8fba-0dda8649d800-kube-api-access-5dp2g\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.823376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823290 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.823376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823327 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.823376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823359 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.823573 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823410 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7zgc\" (UniqueName: \"kubernetes.io/projected/6b7a15e9-d230-4b35-b96f-2e2d4006086e-kube-api-access-l7zgc\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.823625 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823611 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.823661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.823641 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.832940 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.832886 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dp2g\" (UniqueName: \"kubernetes.io/projected/c7b29228-9cec-4a07-8fba-0dda8649d800-kube-api-access-5dp2g\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:57.882054 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.882023 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr"] Sep 19 09:19:57.891343 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.891316 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:57.892860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.892835 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:19:57.895917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.895657 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr"] Sep 19 09:19:57.924448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924418 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7zgc\" (UniqueName: \"kubernetes.io/projected/6b7a15e9-d230-4b35-b96f-2e2d4006086e-kube-api-access-l7zgc\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.924636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924468 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:57.924636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924508 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-864qx\" (UniqueName: \"kubernetes.io/projected/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-kube-api-access-864qx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:57.924636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924545 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.924636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924613 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:57.924839 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924692 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.924875 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924846 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.924951 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.924935 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.934293 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.934247 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7zgc\" (UniqueName: \"kubernetes.io/projected/6b7a15e9-d230-4b35-b96f-2e2d4006086e-kube-api-access-l7zgc\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:57.987807 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:57.987774 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:19:58.025313 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.025265 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:58.025494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.025355 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-864qx\" (UniqueName: \"kubernetes.io/projected/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-kube-api-access-864qx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:58.025494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.025411 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:58.025798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.025776 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:58.025928 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.025907 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:58.035631 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.035301 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-864qx\" (UniqueName: \"kubernetes.io/projected/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-kube-api-access-864qx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:58.088689 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.088624 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:19:58.169202 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.168890 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58"] Sep 19 09:19:58.171020 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:19:58.170986 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50ad76a_468e_4b46_a981_c4c98ce88302.slice/crio-f15e71f19917a48f2b49cec6b62fbb22c2963e61182d6697a1b0e9b4561baa93 WatchSource:0}: Error finding container f15e71f19917a48f2b49cec6b62fbb22c2963e61182d6697a1b0e9b4561baa93: Status 404 returned error can't find the container with id f15e71f19917a48f2b49cec6b62fbb22c2963e61182d6697a1b0e9b4561baa93 Sep 19 09:19:58.201937 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.201916 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:19:58.261292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.261210 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf"] Sep 19 09:19:58.270103 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:19:58.270065 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b29228_9cec_4a07_8fba_0dda8649d800.slice/crio-e86125850a70ac6e7d7cdc939ccb188b4c45f2778415d9fea8c8fe89c435b746 WatchSource:0}: Error finding container e86125850a70ac6e7d7cdc939ccb188b4c45f2778415d9fea8c8fe89c435b746: Status 404 returned error can't find the container with id e86125850a70ac6e7d7cdc939ccb188b4c45f2778415d9fea8c8fe89c435b746 Sep 19 09:19:58.367152 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.367125 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm"] Sep 19 09:19:58.368707 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:19:58.368679 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7a15e9_d230_4b35_b96f_2e2d4006086e.slice/crio-6800dbfb87b0c80a82d4d4b2a8cf5b77dc50b120bdacc0d797d0b990b4c497be WatchSource:0}: Error finding container 6800dbfb87b0c80a82d4d4b2a8cf5b77dc50b120bdacc0d797d0b990b4c497be: Status 404 returned error can't find the container with id 6800dbfb87b0c80a82d4d4b2a8cf5b77dc50b120bdacc0d797d0b990b4c497be Sep 19 09:19:58.481282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.481259 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr"] Sep 19 09:19:58.533296 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:19:58.533265 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f5fa5cf_1288_4ccb_b65d_a6c7e441f074.slice/crio-08dcad7eb2431eef238bda545958fca4f99dc62578c9952668efa687efa344c2 WatchSource:0}: Error finding container 08dcad7eb2431eef238bda545958fca4f99dc62578c9952668efa687efa344c2: Status 404 returned error can't find the container with id 08dcad7eb2431eef238bda545958fca4f99dc62578c9952668efa687efa344c2 Sep 19 09:19:58.757216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.757185 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerID="d388a3729cea51d3be775a8c82754339f01d28945ac246f0016e276a29092555" exitCode=0 Sep 19 09:19:58.757651 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.757262 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" event={"ID":"c7b29228-9cec-4a07-8fba-0dda8649d800","Type":"ContainerDied","Data":"d388a3729cea51d3be775a8c82754339f01d28945ac246f0016e276a29092555"} Sep 19 09:19:58.757651 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.757310 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" event={"ID":"c7b29228-9cec-4a07-8fba-0dda8649d800","Type":"ContainerStarted","Data":"e86125850a70ac6e7d7cdc939ccb188b4c45f2778415d9fea8c8fe89c435b746"} Sep 19 09:19:58.758922 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.758901 2585 generic.go:356] "Generic (PLEG): container finished" podID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerID="2c4e2bc19952626326a4f5b3a4f3f09d90cb734b71d9c8a97b75a77fd4d76037" exitCode=0 Sep 19 09:19:58.759043 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.758990 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" event={"ID":"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074","Type":"ContainerDied","Data":"2c4e2bc19952626326a4f5b3a4f3f09d90cb734b71d9c8a97b75a77fd4d76037"} Sep 19 09:19:58.759119 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.759042 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" event={"ID":"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074","Type":"ContainerStarted","Data":"08dcad7eb2431eef238bda545958fca4f99dc62578c9952668efa687efa344c2"} Sep 19 09:19:58.760609 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.760587 2585 generic.go:356] "Generic (PLEG): container finished" podID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerID="4d2b89abcf2551ddffb28df2eca0581bb445b51a3a97ffd7f02f16c41dfc9d41" exitCode=0 Sep 19 09:19:58.760728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.760613 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" event={"ID":"6b7a15e9-d230-4b35-b96f-2e2d4006086e","Type":"ContainerDied","Data":"4d2b89abcf2551ddffb28df2eca0581bb445b51a3a97ffd7f02f16c41dfc9d41"} Sep 19 09:19:58.760728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.760660 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" event={"ID":"6b7a15e9-d230-4b35-b96f-2e2d4006086e","Type":"ContainerStarted","Data":"6800dbfb87b0c80a82d4d4b2a8cf5b77dc50b120bdacc0d797d0b990b4c497be"} Sep 19 09:19:58.762344 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.762324 2585 generic.go:356] "Generic (PLEG): container finished" podID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerID="e33bdb1752bbfbd6e952f246be1dd6d93de980215d1e46332adf8ef694efee6c" exitCode=0 Sep 19 09:19:58.762436 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.762371 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" event={"ID":"c50ad76a-468e-4b46-a981-c4c98ce88302","Type":"ContainerDied","Data":"e33bdb1752bbfbd6e952f246be1dd6d93de980215d1e46332adf8ef694efee6c"} Sep 19 09:19:58.762436 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:19:58.762404 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" event={"ID":"c50ad76a-468e-4b46-a981-c4c98ce88302","Type":"ContainerStarted","Data":"f15e71f19917a48f2b49cec6b62fbb22c2963e61182d6697a1b0e9b4561baa93"} Sep 19 09:20:00.776467 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.776436 2585 generic.go:356] "Generic (PLEG): container finished" podID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerID="903190ae0877c87e5c742fc4dfd67c68993e64c0675bf088c727826a93797896" exitCode=0 Sep 19 09:20:00.776918 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.776531 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" event={"ID":"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074","Type":"ContainerDied","Data":"903190ae0877c87e5c742fc4dfd67c68993e64c0675bf088c727826a93797896"} Sep 19 09:20:00.778417 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.778393 2585 generic.go:356] "Generic (PLEG): container finished" podID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerID="60f80f430b29084aaf9a8b710c827f0f7517fae683468dc35823bf820c7921ac" exitCode=0 Sep 19 09:20:00.778511 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.778469 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" event={"ID":"6b7a15e9-d230-4b35-b96f-2e2d4006086e","Type":"ContainerDied","Data":"60f80f430b29084aaf9a8b710c827f0f7517fae683468dc35823bf820c7921ac"} Sep 19 09:20:00.780431 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.780411 2585 generic.go:356] "Generic (PLEG): container finished" podID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerID="65b6c513d2c4dbef6b9b50fa372a1d495df7154ebd5985b89902eeb540512c94" exitCode=0 Sep 19 09:20:00.780540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.780483 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" event={"ID":"c50ad76a-468e-4b46-a981-c4c98ce88302","Type":"ContainerDied","Data":"65b6c513d2c4dbef6b9b50fa372a1d495df7154ebd5985b89902eeb540512c94"} Sep 19 09:20:00.782707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.782683 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerID="7c691899f242440a35344d38e26555d81a8a351967e871323287285b68ccfd7f" exitCode=0 Sep 19 09:20:00.782803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:00.782749 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" event={"ID":"c7b29228-9cec-4a07-8fba-0dda8649d800","Type":"ContainerDied","Data":"7c691899f242440a35344d38e26555d81a8a351967e871323287285b68ccfd7f"} Sep 19 09:20:01.789635 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.789594 2585 generic.go:356] "Generic (PLEG): container finished" podID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerID="7caf5826505f5990bd9478807c826c81058612609b46fd717eab610752e9adf3" exitCode=0 Sep 19 09:20:01.790086 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.789682 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" event={"ID":"c7b29228-9cec-4a07-8fba-0dda8649d800","Type":"ContainerDied","Data":"7caf5826505f5990bd9478807c826c81058612609b46fd717eab610752e9adf3"} Sep 19 09:20:01.791877 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.791852 2585 generic.go:356] "Generic (PLEG): container finished" podID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerID="7ad0ddc2d4823c6c3f680c6d6b8cd49ef8c67028f7fbdf4d12a6d3a26849b30c" exitCode=0 Sep 19 09:20:01.791991 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.791928 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" event={"ID":"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074","Type":"ContainerDied","Data":"7ad0ddc2d4823c6c3f680c6d6b8cd49ef8c67028f7fbdf4d12a6d3a26849b30c"} Sep 19 09:20:01.794034 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.794012 2585 generic.go:356] "Generic (PLEG): container finished" podID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerID="66678bc778f97736a30bf5bbb07691679262c1ef342d227f773578f8b901aacd" exitCode=0 Sep 19 09:20:01.794124 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.794073 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" event={"ID":"6b7a15e9-d230-4b35-b96f-2e2d4006086e","Type":"ContainerDied","Data":"66678bc778f97736a30bf5bbb07691679262c1ef342d227f773578f8b901aacd"} Sep 19 09:20:01.796087 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.796069 2585 generic.go:356] "Generic (PLEG): container finished" podID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerID="cc1a3ad17356a2f268d0424d4255b457847e94651fca8b39c1174f12f6162d8c" exitCode=0 Sep 19 09:20:01.796150 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:01.796094 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" event={"ID":"c50ad76a-468e-4b46-a981-c4c98ce88302","Type":"ContainerDied","Data":"cc1a3ad17356a2f268d0424d4255b457847e94651fca8b39c1174f12f6162d8c"} Sep 19 09:20:02.971968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:02.971915 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:20:02.975048 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:02.975027 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:20:03.001753 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.001728 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:20:03.010166 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.010145 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:20:03.067405 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067377 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-864qx\" (UniqueName: \"kubernetes.io/projected/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-kube-api-access-864qx\") pod \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " Sep 19 09:20:03.067591 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067428 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-util\") pod \"c50ad76a-468e-4b46-a981-c4c98ce88302\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " Sep 19 09:20:03.067591 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067452 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7zgc\" (UniqueName: \"kubernetes.io/projected/6b7a15e9-d230-4b35-b96f-2e2d4006086e-kube-api-access-l7zgc\") pod \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " Sep 19 09:20:03.067591 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067561 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dp2g\" (UniqueName: \"kubernetes.io/projected/c7b29228-9cec-4a07-8fba-0dda8649d800-kube-api-access-5dp2g\") pod \"c7b29228-9cec-4a07-8fba-0dda8649d800\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " Sep 19 09:20:03.067767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067601 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-bundle\") pod \"c50ad76a-468e-4b46-a981-c4c98ce88302\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " Sep 19 09:20:03.067767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067623 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-bundle\") pod \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " Sep 19 09:20:03.067767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067661 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rz5j\" (UniqueName: \"kubernetes.io/projected/c50ad76a-468e-4b46-a981-c4c98ce88302-kube-api-access-5rz5j\") pod \"c50ad76a-468e-4b46-a981-c4c98ce88302\" (UID: \"c50ad76a-468e-4b46-a981-c4c98ce88302\") " Sep 19 09:20:03.067767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067690 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-bundle\") pod \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " Sep 19 09:20:03.067767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067714 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-bundle\") pod \"c7b29228-9cec-4a07-8fba-0dda8649d800\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " Sep 19 09:20:03.067767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067746 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-util\") pod \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\" (UID: \"6b7a15e9-d230-4b35-b96f-2e2d4006086e\") " Sep 19 09:20:03.068043 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067784 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-util\") pod \"c7b29228-9cec-4a07-8fba-0dda8649d800\" (UID: \"c7b29228-9cec-4a07-8fba-0dda8649d800\") " Sep 19 09:20:03.068043 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.067806 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-util\") pod \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\" (UID: \"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074\") " Sep 19 09:20:03.069987 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.069347 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-bundle" (OuterVolumeSpecName: "bundle") pod "c50ad76a-468e-4b46-a981-c4c98ce88302" (UID: "c50ad76a-468e-4b46-a981-c4c98ce88302"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.069987 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.069441 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-bundle" (OuterVolumeSpecName: "bundle") pod "c7b29228-9cec-4a07-8fba-0dda8649d800" (UID: "c7b29228-9cec-4a07-8fba-0dda8649d800"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.069987 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.069809 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-bundle" (OuterVolumeSpecName: "bundle") pod "6b7a15e9-d230-4b35-b96f-2e2d4006086e" (UID: "6b7a15e9-d230-4b35-b96f-2e2d4006086e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.070208 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.070074 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-bundle" (OuterVolumeSpecName: "bundle") pod "0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" (UID: "0f5fa5cf-1288-4ccb-b65d-a6c7e441f074"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.071340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.071311 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b7a15e9-d230-4b35-b96f-2e2d4006086e-kube-api-access-l7zgc" (OuterVolumeSpecName: "kube-api-access-l7zgc") pod "6b7a15e9-d230-4b35-b96f-2e2d4006086e" (UID: "6b7a15e9-d230-4b35-b96f-2e2d4006086e"). InnerVolumeSpecName "kube-api-access-l7zgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:03.071340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.071318 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-kube-api-access-864qx" (OuterVolumeSpecName: "kube-api-access-864qx") pod "0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" (UID: "0f5fa5cf-1288-4ccb-b65d-a6c7e441f074"). InnerVolumeSpecName "kube-api-access-864qx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:03.071460 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.071338 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c50ad76a-468e-4b46-a981-c4c98ce88302-kube-api-access-5rz5j" (OuterVolumeSpecName: "kube-api-access-5rz5j") pod "c50ad76a-468e-4b46-a981-c4c98ce88302" (UID: "c50ad76a-468e-4b46-a981-c4c98ce88302"). InnerVolumeSpecName "kube-api-access-5rz5j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:03.071460 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.071349 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b29228-9cec-4a07-8fba-0dda8649d800-kube-api-access-5dp2g" (OuterVolumeSpecName: "kube-api-access-5dp2g") pod "c7b29228-9cec-4a07-8fba-0dda8649d800" (UID: "c7b29228-9cec-4a07-8fba-0dda8649d800"). InnerVolumeSpecName "kube-api-access-5dp2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:03.076262 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.076240 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-util" (OuterVolumeSpecName: "util") pod "c7b29228-9cec-4a07-8fba-0dda8649d800" (UID: "c7b29228-9cec-4a07-8fba-0dda8649d800"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.076687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.076665 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-util" (OuterVolumeSpecName: "util") pod "0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" (UID: "0f5fa5cf-1288-4ccb-b65d-a6c7e441f074"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.077494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.077472 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-util" (OuterVolumeSpecName: "util") pod "c50ad76a-468e-4b46-a981-c4c98ce88302" (UID: "c50ad76a-468e-4b46-a981-c4c98ce88302"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.078046 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.078030 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-util" (OuterVolumeSpecName: "util") pod "6b7a15e9-d230-4b35-b96f-2e2d4006086e" (UID: "6b7a15e9-d230-4b35-b96f-2e2d4006086e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:03.168917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168888 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.168917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168912 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.168917 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168921 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rz5j\" (UniqueName: \"kubernetes.io/projected/c50ad76a-468e-4b46-a981-c4c98ce88302-kube-api-access-5rz5j\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168932 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168941 2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-bundle\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168948 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6b7a15e9-d230-4b35-b96f-2e2d4006086e-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168955 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c7b29228-9cec-4a07-8fba-0dda8649d800-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168963 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168970 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-864qx\" (UniqueName: \"kubernetes.io/projected/0f5fa5cf-1288-4ccb-b65d-a6c7e441f074-kube-api-access-864qx\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168983 2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c50ad76a-468e-4b46-a981-c4c98ce88302-util\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168991 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7zgc\" (UniqueName: \"kubernetes.io/projected/6b7a15e9-d230-4b35-b96f-2e2d4006086e-kube-api-access-l7zgc\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.169132 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.168999 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dp2g\" (UniqueName: \"kubernetes.io/projected/c7b29228-9cec-4a07-8fba-0dda8649d800-kube-api-access-5dp2g\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:03.815582 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.815541 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" event={"ID":"0f5fa5cf-1288-4ccb-b65d-a6c7e441f074","Type":"ContainerDied","Data":"08dcad7eb2431eef238bda545958fca4f99dc62578c9952668efa687efa344c2"} Sep 19 09:20:03.815773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.815628 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08dcad7eb2431eef238bda545958fca4f99dc62578c9952668efa687efa344c2" Sep 19 09:20:03.815773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.815552 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6ngtvr" Sep 19 09:20:03.817973 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.817949 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" event={"ID":"6b7a15e9-d230-4b35-b96f-2e2d4006086e","Type":"ContainerDied","Data":"6800dbfb87b0c80a82d4d4b2a8cf5b77dc50b120bdacc0d797d0b990b4c497be"} Sep 19 09:20:03.817973 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.817963 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aprlgm" Sep 19 09:20:03.818130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.817990 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6800dbfb87b0c80a82d4d4b2a8cf5b77dc50b120bdacc0d797d0b990b4c497be" Sep 19 09:20:03.820360 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.820341 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" Sep 19 09:20:03.820453 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.820354 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7t6v58" event={"ID":"c50ad76a-468e-4b46-a981-c4c98ce88302","Type":"ContainerDied","Data":"f15e71f19917a48f2b49cec6b62fbb22c2963e61182d6697a1b0e9b4561baa93"} Sep 19 09:20:03.820453 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.820389 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f15e71f19917a48f2b49cec6b62fbb22c2963e61182d6697a1b0e9b4561baa93" Sep 19 09:20:03.822443 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.822425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" event={"ID":"c7b29228-9cec-4a07-8fba-0dda8649d800","Type":"ContainerDied","Data":"e86125850a70ac6e7d7cdc939ccb188b4c45f2778415d9fea8c8fe89c435b746"} Sep 19 09:20:03.822576 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.822453 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e86125850a70ac6e7d7cdc939ccb188b4c45f2778415d9fea8c8fe89c435b746" Sep 19 09:20:03.822576 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:03.822564 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckjzwf" Sep 19 09:20:09.563898 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.563869 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-b6wgm"] Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564145 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564155 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564163 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerName="pull" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564170 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerName="pull" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564175 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564180 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564191 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerName="pull" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564197 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerName="pull" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564209 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564214 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564219 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerName="pull" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564224 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerName="pull" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564230 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564235 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564246 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564250 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564255 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564260 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564268 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564273 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerName="extract" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564279 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564283 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerName="util" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564288 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerName="pull" Sep 19 09:20:09.564319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564293 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerName="pull" Sep 19 09:20:09.565092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564341 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f5fa5cf-1288-4ccb-b65d-a6c7e441f074" containerName="extract" Sep 19 09:20:09.565092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564347 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c50ad76a-468e-4b46-a981-c4c98ce88302" containerName="extract" Sep 19 09:20:09.565092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564354 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b7a15e9-d230-4b35-b96f-2e2d4006086e" containerName="extract" Sep 19 09:20:09.565092 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.564359 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b29228-9cec-4a07-8fba-0dda8649d800" containerName="extract" Sep 19 09:20:09.568667 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.568652 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" Sep 19 09:20:09.576621 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.576603 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 19 09:20:09.577087 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.577073 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-dnt7f\"" Sep 19 09:20:09.577361 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.577344 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 19 09:20:09.588671 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.588650 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-b6wgm"] Sep 19 09:20:09.622046 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.622020 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xrck\" (UniqueName: \"kubernetes.io/projected/61cd96ef-9cb3-47ec-b570-365e15622415-kube-api-access-2xrck\") pod \"authorino-operator-76b478bd78-b6wgm\" (UID: \"61cd96ef-9cb3-47ec-b570-365e15622415\") " pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" Sep 19 09:20:09.722346 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.722323 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xrck\" (UniqueName: \"kubernetes.io/projected/61cd96ef-9cb3-47ec-b570-365e15622415-kube-api-access-2xrck\") pod \"authorino-operator-76b478bd78-b6wgm\" (UID: \"61cd96ef-9cb3-47ec-b570-365e15622415\") " pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" Sep 19 09:20:09.737490 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.734976 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xrck\" (UniqueName: \"kubernetes.io/projected/61cd96ef-9cb3-47ec-b570-365e15622415-kube-api-access-2xrck\") pod \"authorino-operator-76b478bd78-b6wgm\" (UID: \"61cd96ef-9cb3-47ec-b570-365e15622415\") " pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" Sep 19 09:20:09.878147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:09.878096 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" Sep 19 09:20:10.155870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:10.155799 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-b6wgm"] Sep 19 09:20:10.158942 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:10.158915 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cd96ef_9cb3_47ec_b570_365e15622415.slice/crio-0c8fdd39c85ceb0513d823228e42dde7bc091f7d5178091f1968ac8e00faf3c4 WatchSource:0}: Error finding container 0c8fdd39c85ceb0513d823228e42dde7bc091f7d5178091f1968ac8e00faf3c4: Status 404 returned error can't find the container with id 0c8fdd39c85ceb0513d823228e42dde7bc091f7d5178091f1968ac8e00faf3c4 Sep 19 09:20:10.856844 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:10.856813 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" event={"ID":"61cd96ef-9cb3-47ec-b570-365e15622415","Type":"ContainerStarted","Data":"0c8fdd39c85ceb0513d823228e42dde7bc091f7d5178091f1968ac8e00faf3c4"} Sep 19 09:20:11.818095 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:11.818065 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x"] Sep 19 09:20:11.824625 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:11.824598 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:11.827113 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:11.827087 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2qd4d\"" Sep 19 09:20:11.838091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:11.838068 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x"] Sep 19 09:20:11.942532 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:11.942480 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjv9b\" (UniqueName: \"kubernetes.io/projected/cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0-kube-api-access-fjv9b\") pod \"limitador-operator-controller-manager-74495c69f8-ltg5x\" (UID: \"cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:12.043423 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:12.043393 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjv9b\" (UniqueName: \"kubernetes.io/projected/cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0-kube-api-access-fjv9b\") pod \"limitador-operator-controller-manager-74495c69f8-ltg5x\" (UID: \"cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:12.056459 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:12.056426 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjv9b\" (UniqueName: \"kubernetes.io/projected/cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0-kube-api-access-fjv9b\") pod \"limitador-operator-controller-manager-74495c69f8-ltg5x\" (UID: \"cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:12.137598 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:12.137507 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:12.867963 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:12.867897 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" event={"ID":"61cd96ef-9cb3-47ec-b570-365e15622415","Type":"ContainerStarted","Data":"833d3df72028a0566a34e9a5b69228a2b2dc158aca77fa0b565965b4ceab1c56"} Sep 19 09:20:12.868098 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:12.868036 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" Sep 19 09:20:12.920610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:12.920563 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" podStartSLOduration=1.398849681 podStartE2EDuration="3.920547708s" totalImagesPullingTime="2.521698027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:12.918979607 +0000 UTC m=+819.677100634" watchObservedRunningTime="2026-09-19 09:20:12.920547708 +0000 UTC m=+819.678668732" Sep 19 09:20:12.924397 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:12.924370 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x"] Sep 19 09:20:12.926028 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:12.926000 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf450e3d_e1a2_4abc_9b93_cfd9b31c0eb0.slice/crio-f80ff9b3284c1e1c91afd53e6bbbf46d00a250225371703cee351b6ab17b0aea WatchSource:0}: Error finding container f80ff9b3284c1e1c91afd53e6bbbf46d00a250225371703cee351b6ab17b0aea: Status 404 returned error can't find the container with id f80ff9b3284c1e1c91afd53e6bbbf46d00a250225371703cee351b6ab17b0aea Sep 19 09:20:13.874687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:13.874651 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" event={"ID":"cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0","Type":"ContainerStarted","Data":"f80ff9b3284c1e1c91afd53e6bbbf46d00a250225371703cee351b6ab17b0aea"} Sep 19 09:20:14.162554 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.162467 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r"] Sep 19 09:20:14.166038 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.166017 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" Sep 19 09:20:14.168570 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.168539 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 19 09:20:14.168901 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.168874 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-vv8c2\"" Sep 19 09:20:14.185724 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.185703 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r"] Sep 19 09:20:14.259101 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.259066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9hwb\" (UniqueName: \"kubernetes.io/projected/14bebdf9-5b8e-4cb7-8467-9d45a6544e1b-kube-api-access-g9hwb\") pod \"dns-operator-controller-manager-7b84cd7c9c-wtv9r\" (UID: \"14bebdf9-5b8e-4cb7-8467-9d45a6544e1b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" Sep 19 09:20:14.359771 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.359736 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9hwb\" (UniqueName: \"kubernetes.io/projected/14bebdf9-5b8e-4cb7-8467-9d45a6544e1b-kube-api-access-g9hwb\") pod \"dns-operator-controller-manager-7b84cd7c9c-wtv9r\" (UID: \"14bebdf9-5b8e-4cb7-8467-9d45a6544e1b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" Sep 19 09:20:14.374062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.374038 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9hwb\" (UniqueName: \"kubernetes.io/projected/14bebdf9-5b8e-4cb7-8467-9d45a6544e1b-kube-api-access-g9hwb\") pod \"dns-operator-controller-manager-7b84cd7c9c-wtv9r\" (UID: \"14bebdf9-5b8e-4cb7-8467-9d45a6544e1b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" Sep 19 09:20:14.477766 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.477742 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" Sep 19 09:20:14.792707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.792585 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r"] Sep 19 09:20:14.795071 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:14.795044 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14bebdf9_5b8e_4cb7_8467_9d45a6544e1b.slice/crio-626670d1beb2cad708d278d3ee327a70d2eb941bce1506361ef2980986734dd4 WatchSource:0}: Error finding container 626670d1beb2cad708d278d3ee327a70d2eb941bce1506361ef2980986734dd4: Status 404 returned error can't find the container with id 626670d1beb2cad708d278d3ee327a70d2eb941bce1506361ef2980986734dd4 Sep 19 09:20:14.880978 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.880941 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" event={"ID":"cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0","Type":"ContainerStarted","Data":"1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c"} Sep 19 09:20:14.881315 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.881051 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:14.882221 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.882200 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" event={"ID":"14bebdf9-5b8e-4cb7-8467-9d45a6544e1b","Type":"ContainerStarted","Data":"626670d1beb2cad708d278d3ee327a70d2eb941bce1506361ef2980986734dd4"} Sep 19 09:20:14.904417 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:14.904379 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" podStartSLOduration=2.391149736 podStartE2EDuration="3.904367994s" totalImagesPullingTime="1.513218258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:14.904094113 +0000 UTC m=+821.662215140" watchObservedRunningTime="2026-09-19 09:20:14.904367994 +0000 UTC m=+821.662489019" Sep 19 09:20:16.894281 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:16.894244 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" event={"ID":"14bebdf9-5b8e-4cb7-8467-9d45a6544e1b","Type":"ContainerStarted","Data":"b9ad834a1f9862444d1e3395e4afb985fbe5b8bc6fb998533492ca988fc40be7"} Sep 19 09:20:16.894608 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:16.894309 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" Sep 19 09:20:16.921962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:16.921925 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" podStartSLOduration=1.044875654 podStartE2EDuration="2.921911542s" totalImagesPullingTime="1.877035888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:16.920731284 +0000 UTC m=+823.678852309" watchObservedRunningTime="2026-09-19 09:20:16.921911542 +0000 UTC m=+823.680032566" Sep 19 09:20:18.805754 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.805720 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q"] Sep 19 09:20:18.809227 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.809207 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:18.812928 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.812911 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x86cm\"" Sep 19 09:20:18.833557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.833530 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q"] Sep 19 09:20:18.894467 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.894432 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxzgz\" (UniqueName: \"kubernetes.io/projected/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-kube-api-access-cxzgz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:18.894599 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.894474 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:18.995376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.995341 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxzgz\" (UniqueName: \"kubernetes.io/projected/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-kube-api-access-cxzgz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:18.995554 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.995383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:18.995746 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:18.995729 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:19.007679 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:19.007648 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxzgz\" (UniqueName: \"kubernetes.io/projected/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-kube-api-access-cxzgz\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:19.119026 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:19.118953 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:19.413073 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:19.413043 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q"] Sep 19 09:20:19.417721 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:19.417684 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8bbb169_54b0_4acd_95ff_d67e3fd3ab2b.slice/crio-2f843e48acd745f7fda3532ed8a7a3d0bde62b5e0a6a623d4c7933fa1943e521 WatchSource:0}: Error finding container 2f843e48acd745f7fda3532ed8a7a3d0bde62b5e0a6a623d4c7933fa1943e521: Status 404 returned error can't find the container with id 2f843e48acd745f7fda3532ed8a7a3d0bde62b5e0a6a623d4c7933fa1943e521 Sep 19 09:20:19.910557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:19.910500 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" event={"ID":"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b","Type":"ContainerStarted","Data":"2f843e48acd745f7fda3532ed8a7a3d0bde62b5e0a6a623d4c7933fa1943e521"} Sep 19 09:20:23.878018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:23.877988 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-b6wgm" Sep 19 09:20:23.935038 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:23.935000 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" event={"ID":"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b","Type":"ContainerStarted","Data":"229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df"} Sep 19 09:20:23.935223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:23.935157 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:23.955331 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:23.955282 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" podStartSLOduration=2.21001306 podStartE2EDuration="5.955267785s" totalImagesPullingTime="3.745254725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:23.954182169 +0000 UTC m=+830.712303195" watchObservedRunningTime="2026-09-19 09:20:23.955267785 +0000 UTC m=+830.713388807" Sep 19 09:20:25.890280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:25.890244 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:27.901415 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:27.901380 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wtv9r" Sep 19 09:20:34.946444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:34.946406 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:36.467424 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.467388 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w"] Sep 19 09:20:36.472393 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.472374 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:36.495215 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.495190 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w"] Sep 19 09:20:36.550550 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.550490 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mz68\" (UniqueName: \"kubernetes.io/projected/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-kube-api-access-8mz68\") pod \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:36.550715 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.550605 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:36.651365 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.651325 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mz68\" (UniqueName: \"kubernetes.io/projected/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-kube-api-access-8mz68\") pod \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:36.651578 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.651383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:36.651743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.651726 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:36.661781 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.661755 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mz68\" (UniqueName: \"kubernetes.io/projected/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-kube-api-access-8mz68\") pod \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:36.782686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:36.782583 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:37.057082 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.057056 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w"] Sep 19 09:20:37.059755 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:37.059707 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53a4cc4_220f_4cff_aa1a_4f1a30796bf6.slice/crio-9b283c87fbb3da349def9cf32c40c07a7ea846d2a6587911d53ffc44d606509e WatchSource:0}: Error finding container 9b283c87fbb3da349def9cf32c40c07a7ea846d2a6587911d53ffc44d606509e: Status 404 returned error can't find the container with id 9b283c87fbb3da349def9cf32c40c07a7ea846d2a6587911d53ffc44d606509e Sep 19 09:20:37.145625 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.145581 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w"] Sep 19 09:20:37.151891 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.151863 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w"] Sep 19 09:20:37.163008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.162987 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q"] Sep 19 09:20:37.163177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.163154 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" containerName="manager" containerID="cri-o://229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df" gracePeriod=2 Sep 19 09:20:37.167608 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.167583 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf"] Sep 19 09:20:37.167992 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.167957 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" containerName="manager" Sep 19 09:20:37.168244 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.168226 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" containerName="manager" Sep 19 09:20:37.168379 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.168364 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" containerName="manager" Sep 19 09:20:37.172178 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.172156 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.175006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.174972 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q"] Sep 19 09:20:37.176140 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.176107 2585 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\": pods \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:37.186909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.186885 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf"] Sep 19 09:20:37.188752 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.188728 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x"] Sep 19 09:20:37.188971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.188932 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" containerName="manager" containerID="cri-o://1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c" gracePeriod=2 Sep 19 09:20:37.194381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.194351 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c"] Sep 19 09:20:37.194821 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.194804 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" containerName="manager" Sep 19 09:20:37.194821 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.194823 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" containerName="manager" Sep 19 09:20:37.194954 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.194902 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" containerName="manager" Sep 19 09:20:37.198363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.198343 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.204188 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.204167 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x"] Sep 19 09:20:37.208742 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.208713 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:37.210897 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.210867 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:37.223745 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.222895 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ltg5x\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:37.228262 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.228234 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c"] Sep 19 09:20:37.230313 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.230289 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5"] Sep 19 09:20:37.230624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.230611 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" containerName="manager" Sep 19 09:20:37.230624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.230625 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" containerName="manager" Sep 19 09:20:37.230757 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.230687 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" containerName="manager" Sep 19 09:20:37.233575 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.233556 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" Sep 19 09:20:37.242069 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.242045 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5"] Sep 19 09:20:37.258160 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.258131 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7rvf\" (UniqueName: \"kubernetes.io/projected/904747b9-652d-4e0f-8d8f-170434f63b05-kube-api-access-p7rvf\") pod \"kuadrant-operator-controller-manager-5f685c496f-tp8wf\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.258281 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.258249 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/904747b9-652d-4e0f-8d8f-170434f63b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-tp8wf\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.275682 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.275635 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:37.277698 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.277670 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ltg5x\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:37.359175 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.359137 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7rvf\" (UniqueName: \"kubernetes.io/projected/904747b9-652d-4e0f-8d8f-170434f63b05-kube-api-access-p7rvf\") pod \"kuadrant-operator-controller-manager-5f685c496f-tp8wf\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.359175 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.359193 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b0c593-6a84-415e-9823-e9db5107c002-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wlb9c\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.359487 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.359234 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpjd9\" (UniqueName: \"kubernetes.io/projected/1b78eae0-0e76-442b-996d-4219653d309e-kube-api-access-kpjd9\") pod \"limitador-operator-controller-manager-74495c69f8-5fsj5\" (UID: \"1b78eae0-0e76-442b-996d-4219653d309e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" Sep 19 09:20:37.359487 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.359287 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbxxc\" (UniqueName: \"kubernetes.io/projected/56b0c593-6a84-415e-9823-e9db5107c002-kube-api-access-zbxxc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wlb9c\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.359487 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.359332 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/904747b9-652d-4e0f-8d8f-170434f63b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-tp8wf\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.359732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.359683 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/904747b9-652d-4e0f-8d8f-170434f63b05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-tp8wf\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.370734 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.370689 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7rvf\" (UniqueName: \"kubernetes.io/projected/904747b9-652d-4e0f-8d8f-170434f63b05-kube-api-access-p7rvf\") pod \"kuadrant-operator-controller-manager-5f685c496f-tp8wf\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.412320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.412295 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:37.414690 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.414660 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:37.416493 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.416474 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ltg5x\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:37.419982 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.419962 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:37.422037 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.422015 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xfp8q\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:37.423784 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:37.423762 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-ltg5x\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:37.460497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.460469 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b0c593-6a84-415e-9823-e9db5107c002-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wlb9c\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.460662 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.460502 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpjd9\" (UniqueName: \"kubernetes.io/projected/1b78eae0-0e76-442b-996d-4219653d309e-kube-api-access-kpjd9\") pod \"limitador-operator-controller-manager-74495c69f8-5fsj5\" (UID: \"1b78eae0-0e76-442b-996d-4219653d309e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" Sep 19 09:20:37.460662 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.460541 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbxxc\" (UniqueName: \"kubernetes.io/projected/56b0c593-6a84-415e-9823-e9db5107c002-kube-api-access-zbxxc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wlb9c\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.460849 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.460830 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b0c593-6a84-415e-9823-e9db5107c002-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wlb9c\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.471960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.471927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbxxc\" (UniqueName: \"kubernetes.io/projected/56b0c593-6a84-415e-9823-e9db5107c002-kube-api-access-zbxxc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wlb9c\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.471960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.471927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpjd9\" (UniqueName: \"kubernetes.io/projected/1b78eae0-0e76-442b-996d-4219653d309e-kube-api-access-kpjd9\") pod \"limitador-operator-controller-manager-74495c69f8-5fsj5\" (UID: \"1b78eae0-0e76-442b-996d-4219653d309e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" Sep 19 09:20:37.561123 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.561088 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-extensions-socket-volume\") pod \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " Sep 19 09:20:37.561279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.561149 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxzgz\" (UniqueName: \"kubernetes.io/projected/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-kube-api-access-cxzgz\") pod \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\" (UID: \"c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b\") " Sep 19 09:20:37.561279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.561185 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjv9b\" (UniqueName: \"kubernetes.io/projected/cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0-kube-api-access-fjv9b\") pod \"cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0\" (UID: \"cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0\") " Sep 19 09:20:37.561456 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.561433 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" (UID: "c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:37.563354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.563331 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0-kube-api-access-fjv9b" (OuterVolumeSpecName: "kube-api-access-fjv9b") pod "cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" (UID: "cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0"). InnerVolumeSpecName "kube-api-access-fjv9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:37.563461 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.563365 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-kube-api-access-cxzgz" (OuterVolumeSpecName: "kube-api-access-cxzgz") pod "c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" (UID: "c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b"). InnerVolumeSpecName "kube-api-access-cxzgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:37.574500 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.574479 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:37.581619 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.581597 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:37.588508 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.588487 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" Sep 19 09:20:37.662084 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.662058 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-extensions-socket-volume\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:37.662084 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.662084 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxzgz\" (UniqueName: \"kubernetes.io/projected/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b-kube-api-access-cxzgz\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:37.662218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.662096 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjv9b\" (UniqueName: \"kubernetes.io/projected/cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0-kube-api-access-fjv9b\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:37.895619 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:37.895533 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904747b9_652d_4e0f_8d8f_170434f63b05.slice/crio-08e246bb36abb49b4e470dfb118aa0f9a0c3fc94ee1461991cb76dcdb39ce07b WatchSource:0}: Error finding container 08e246bb36abb49b4e470dfb118aa0f9a0c3fc94ee1461991cb76dcdb39ce07b: Status 404 returned error can't find the container with id 08e246bb36abb49b4e470dfb118aa0f9a0c3fc94ee1461991cb76dcdb39ce07b Sep 19 09:20:37.896661 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:37.896631 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b0c593_6a84_415e_9823_e9db5107c002.slice/crio-e4c82b747f6b0f072ba5b2e0a9900a199731266e2643881e2e9299af4401feb8 WatchSource:0}: Error finding container e4c82b747f6b0f072ba5b2e0a9900a199731266e2643881e2e9299af4401feb8: Status 404 returned error can't find the container with id e4c82b747f6b0f072ba5b2e0a9900a199731266e2643881e2e9299af4401feb8 Sep 19 09:20:37.897696 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:37.897670 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b78eae0_0e76_442b_996d_4219653d309e.slice/crio-839e0787ad8204ae7d465af443232a8a4dd148241c480e296fe739d8889508fc WatchSource:0}: Error finding container 839e0787ad8204ae7d465af443232a8a4dd148241c480e296fe739d8889508fc: Status 404 returned error can't find the container with id 839e0787ad8204ae7d465af443232a8a4dd148241c480e296fe739d8889508fc Sep 19 09:20:37.899314 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.899292 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5"] Sep 19 09:20:37.901690 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.901670 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf"] Sep 19 09:20:37.903618 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:37.903595 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c"] Sep 19 09:20:38.007929 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.007887 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" event={"ID":"1b78eae0-0e76-442b-996d-4219653d309e","Type":"ContainerStarted","Data":"becff27527b91cff45cde78ad38bf8c52ca313a74919d3844ecdd37ee3f263fa"} Sep 19 09:20:38.008074 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.007945 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" event={"ID":"1b78eae0-0e76-442b-996d-4219653d309e","Type":"ContainerStarted","Data":"839e0787ad8204ae7d465af443232a8a4dd148241c480e296fe739d8889508fc"} Sep 19 09:20:38.008074 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.007990 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" Sep 19 09:20:38.009753 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:38.009731 2585 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" container="manager" Sep 19 09:20:38.011345 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.011321 2585 generic.go:356] "Generic (PLEG): container finished" podID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" containerID="229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df" exitCode=0 Sep 19 09:20:38.011449 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.011379 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xfp8q" Sep 19 09:20:38.011449 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.011410 2585 scope.go:117] "RemoveContainer" containerID="229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df" Sep 19 09:20:38.014404 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.013874 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" event={"ID":"904747b9-652d-4e0f-8d8f-170434f63b05","Type":"ContainerStarted","Data":"a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647"} Sep 19 09:20:38.014404 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.013935 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" event={"ID":"904747b9-652d-4e0f-8d8f-170434f63b05","Type":"ContainerStarted","Data":"08e246bb36abb49b4e470dfb118aa0f9a0c3fc94ee1461991cb76dcdb39ce07b"} Sep 19 09:20:38.014404 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.013966 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:38.015792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.015769 2585 generic.go:356] "Generic (PLEG): container finished" podID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" containerID="1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c" exitCode=0 Sep 19 09:20:38.016872 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.015907 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-ltg5x" Sep 19 09:20:38.019575 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.019428 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" event={"ID":"56b0c593-6a84-415e-9823-e9db5107c002","Type":"ContainerStarted","Data":"0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab"} Sep 19 09:20:38.019575 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.019480 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" event={"ID":"56b0c593-6a84-415e-9823-e9db5107c002","Type":"ContainerStarted","Data":"e4c82b747f6b0f072ba5b2e0a9900a199731266e2643881e2e9299af4401feb8"} Sep 19 09:20:38.023258 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.023235 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:38.024674 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.024443 2585 scope.go:117] "RemoveContainer" containerID="229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df" Sep 19 09:20:38.024818 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:38.024782 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df\": container with ID starting with 229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df not found: ID does not exist" containerID="229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df" Sep 19 09:20:38.024865 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.024814 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df"} err="failed to get container status \"229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df\": rpc error: code = NotFound desc = could not find container \"229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df\": container with ID starting with 229fcda14955616fc0e9c101931517196fcdc6070431b2f634c9ced808ca88df not found: ID does not exist" Sep 19 09:20:38.024865 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.024838 2585 scope.go:117] "RemoveContainer" containerID="1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c" Sep 19 09:20:38.036088 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.036073 2585 scope.go:117] "RemoveContainer" containerID="1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c" Sep 19 09:20:38.036341 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:38.036324 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c\": container with ID starting with 1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c not found: ID does not exist" containerID="1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c" Sep 19 09:20:38.036383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.036351 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c"} err="failed to get container status \"1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c\": rpc error: code = NotFound desc = could not find container \"1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c\": container with ID starting with 1d8d9852913c2d55fbd8b91412e641c397e701bff43174660fe7b9004970033c not found: ID does not exist" Sep 19 09:20:38.046384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.046346 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" podStartSLOduration=1.046332065 podStartE2EDuration="1.046332065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:38.044776019 +0000 UTC m=+844.802897045" watchObservedRunningTime="2026-09-19 09:20:38.046332065 +0000 UTC m=+844.804453090" Sep 19 09:20:38.070640 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.070597 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" podStartSLOduration=1.070582935 podStartE2EDuration="1.070582935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:38.067556088 +0000 UTC m=+844.825677112" watchObservedRunningTime="2026-09-19 09:20:38.070582935 +0000 UTC m=+844.828704081" Sep 19 09:20:38.093457 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:38.093407 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" podStartSLOduration=1.093392345 podStartE2EDuration="1.093392345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:38.09194007 +0000 UTC m=+844.850061092" watchObservedRunningTime="2026-09-19 09:20:38.093392345 +0000 UTC m=+844.851513369" Sep 19 09:20:39.029977 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.029930 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" containerName="manager" containerID="cri-o://be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd" gracePeriod=2 Sep 19 09:20:39.032315 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:39.032283 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:39.277831 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.277808 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:39.280229 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:39.280177 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:39.377916 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.377889 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mz68\" (UniqueName: \"kubernetes.io/projected/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-kube-api-access-8mz68\") pod \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " Sep 19 09:20:39.378049 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.377926 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-extensions-socket-volume\") pod \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\" (UID: \"e53a4cc4-220f-4cff-aa1a-4f1a30796bf6\") " Sep 19 09:20:39.378237 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.378206 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" (UID: "e53a4cc4-220f-4cff-aa1a-4f1a30796bf6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:39.380003 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.379976 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-kube-api-access-8mz68" (OuterVolumeSpecName: "kube-api-access-8mz68") pod "e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" (UID: "e53a4cc4-220f-4cff-aa1a-4f1a30796bf6"). InnerVolumeSpecName "kube-api-access-8mz68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:39.436484 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.436438 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b" path="/var/lib/kubelet/pods/c8bbb169-54b0-4acd-95ff-d67e3fd3ab2b/volumes" Sep 19 09:20:39.436818 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.436804 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0" path="/var/lib/kubelet/pods/cf450e3d-e1a2-4abc-9b93-cfd9b31c0eb0/volumes" Sep 19 09:20:39.437121 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.437109 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" path="/var/lib/kubelet/pods/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6/volumes" Sep 19 09:20:39.478700 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.478669 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mz68\" (UniqueName: \"kubernetes.io/projected/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-kube-api-access-8mz68\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:39.478700 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:39.478698 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e53a4cc4-220f-4cff-aa1a-4f1a30796bf6-extensions-socket-volume\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:40.036640 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:40.036599 2585 generic.go:356] "Generic (PLEG): container finished" podID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" containerID="be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd" exitCode=2 Sep 19 09:20:40.036640 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:40.036652 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:40.037176 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:40.036706 2585 scope.go:117] "RemoveContainer" containerID="be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd" Sep 19 09:20:40.041454 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:40.041424 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-bzz7w\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="e53a4cc4-220f-4cff-aa1a-4f1a30796bf6" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-bzz7w" Sep 19 09:20:40.045380 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:40.045360 2585 scope.go:117] "RemoveContainer" containerID="be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd" Sep 19 09:20:40.045658 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:40.045637 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd\": container with ID starting with be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd not found: ID does not exist" containerID="be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd" Sep 19 09:20:40.045725 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:40.045666 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd"} err="failed to get container status \"be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd\": rpc error: code = NotFound desc = could not find container \"be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd\": container with ID starting with be26a4216d7761dd59c8b6a04fa4b9f405fb822e3537a8dda21ba6bb4e3b7fbd not found: ID does not exist" Sep 19 09:20:49.032581 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.032546 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:20:49.032942 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.032636 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:49.032942 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.032802 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-5fsj5" Sep 19 09:20:49.112979 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.112937 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c"] Sep 19 09:20:49.113207 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.113181 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" podUID="56b0c593-6a84-415e-9823-e9db5107c002" containerName="manager" containerID="cri-o://0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab" gracePeriod=10 Sep 19 09:20:49.350099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.350076 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:49.365170 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.365151 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbxxc\" (UniqueName: \"kubernetes.io/projected/56b0c593-6a84-415e-9823-e9db5107c002-kube-api-access-zbxxc\") pod \"56b0c593-6a84-415e-9823-e9db5107c002\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " Sep 19 09:20:49.365247 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.365193 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b0c593-6a84-415e-9823-e9db5107c002-extensions-socket-volume\") pod \"56b0c593-6a84-415e-9823-e9db5107c002\" (UID: \"56b0c593-6a84-415e-9823-e9db5107c002\") " Sep 19 09:20:49.365578 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.365555 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56b0c593-6a84-415e-9823-e9db5107c002-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "56b0c593-6a84-415e-9823-e9db5107c002" (UID: "56b0c593-6a84-415e-9823-e9db5107c002"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:20:49.367091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.367072 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56b0c593-6a84-415e-9823-e9db5107c002-kube-api-access-zbxxc" (OuterVolumeSpecName: "kube-api-access-zbxxc") pod "56b0c593-6a84-415e-9823-e9db5107c002" (UID: "56b0c593-6a84-415e-9823-e9db5107c002"). InnerVolumeSpecName "kube-api-access-zbxxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:20:49.466120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.466091 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbxxc\" (UniqueName: \"kubernetes.io/projected/56b0c593-6a84-415e-9823-e9db5107c002-kube-api-access-zbxxc\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:49.466120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.466116 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b0c593-6a84-415e-9823-e9db5107c002-extensions-socket-volume\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:20:49.479851 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.479826 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv"] Sep 19 09:20:49.480154 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.480143 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56b0c593-6a84-415e-9823-e9db5107c002" containerName="manager" Sep 19 09:20:49.480193 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.480156 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="56b0c593-6a84-415e-9823-e9db5107c002" containerName="manager" Sep 19 09:20:49.480226 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.480220 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="56b0c593-6a84-415e-9823-e9db5107c002" containerName="manager" Sep 19 09:20:49.482404 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.482385 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:49.509466 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.509439 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv"] Sep 19 09:20:49.567286 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.567251 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d967\" (UniqueName: \"kubernetes.io/projected/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-kube-api-access-4d967\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w9qpv\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:49.567445 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.567335 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w9qpv\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:49.668289 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.668207 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d967\" (UniqueName: \"kubernetes.io/projected/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-kube-api-access-4d967\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w9qpv\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:49.668289 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.668273 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w9qpv\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:49.668697 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.668676 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w9qpv\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:49.677071 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.677047 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d967\" (UniqueName: \"kubernetes.io/projected/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-kube-api-access-4d967\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w9qpv\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:49.792815 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:49.792779 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:50.068062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.068034 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv"] Sep 19 09:20:50.070821 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:20:50.070793 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a3cf710_71ea_4f2c_b201_3eb573a4a2a3.slice/crio-6208d06f13f3335c489cb72be9f279a07b49b68a37eda593498efc4c5121b458 WatchSource:0}: Error finding container 6208d06f13f3335c489cb72be9f279a07b49b68a37eda593498efc4c5121b458: Status 404 returned error can't find the container with id 6208d06f13f3335c489cb72be9f279a07b49b68a37eda593498efc4c5121b458 Sep 19 09:20:50.098428 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.098401 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" event={"ID":"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3","Type":"ContainerStarted","Data":"6208d06f13f3335c489cb72be9f279a07b49b68a37eda593498efc4c5121b458"} Sep 19 09:20:50.099859 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.099839 2585 generic.go:356] "Generic (PLEG): container finished" podID="56b0c593-6a84-415e-9823-e9db5107c002" containerID="0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab" exitCode=0 Sep 19 09:20:50.099947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.099887 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" event={"ID":"56b0c593-6a84-415e-9823-e9db5107c002","Type":"ContainerDied","Data":"0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab"} Sep 19 09:20:50.099947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.099897 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" Sep 19 09:20:50.099947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.099920 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c" event={"ID":"56b0c593-6a84-415e-9823-e9db5107c002","Type":"ContainerDied","Data":"e4c82b747f6b0f072ba5b2e0a9900a199731266e2643881e2e9299af4401feb8"} Sep 19 09:20:50.099947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.099940 2585 scope.go:117] "RemoveContainer" containerID="0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab" Sep 19 09:20:50.108997 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.108975 2585 scope.go:117] "RemoveContainer" containerID="0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab" Sep 19 09:20:50.109325 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:20:50.109303 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab\": container with ID starting with 0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab not found: ID does not exist" containerID="0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab" Sep 19 09:20:50.109421 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.109331 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab"} err="failed to get container status \"0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab\": rpc error: code = NotFound desc = could not find container \"0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab\": container with ID starting with 0c00b131e40434eba6ea5584e817767dd47a02f293b77402067bff42229e63ab not found: ID does not exist" Sep 19 09:20:50.122481 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.122455 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c"] Sep 19 09:20:50.129573 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:50.129554 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wlb9c"] Sep 19 09:20:51.107829 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:51.107793 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" event={"ID":"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3","Type":"ContainerStarted","Data":"5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306"} Sep 19 09:20:51.108178 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:51.107863 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:20:51.133095 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:51.133048 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" podStartSLOduration=2.133033609 podStartE2EDuration="2.133033609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:51.131278931 +0000 UTC m=+857.889399955" watchObservedRunningTime="2026-09-19 09:20:51.133033609 +0000 UTC m=+857.891154633" Sep 19 09:20:51.436209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:20:51.436129 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56b0c593-6a84-415e-9823-e9db5107c002" path="/var/lib/kubelet/pods/56b0c593-6a84-415e-9823-e9db5107c002/volumes" Sep 19 09:21:02.115142 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.115115 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:21:02.185117 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.185089 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf"] Sep 19 09:21:02.185418 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.185298 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" podUID="904747b9-652d-4e0f-8d8f-170434f63b05" containerName="manager" containerID="cri-o://a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647" gracePeriod=10 Sep 19 09:21:02.424627 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.424600 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:21:02.459983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.459956 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/904747b9-652d-4e0f-8d8f-170434f63b05-extensions-socket-volume\") pod \"904747b9-652d-4e0f-8d8f-170434f63b05\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " Sep 19 09:21:02.460117 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.460016 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7rvf\" (UniqueName: \"kubernetes.io/projected/904747b9-652d-4e0f-8d8f-170434f63b05-kube-api-access-p7rvf\") pod \"904747b9-652d-4e0f-8d8f-170434f63b05\" (UID: \"904747b9-652d-4e0f-8d8f-170434f63b05\") " Sep 19 09:21:02.460405 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.460377 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/904747b9-652d-4e0f-8d8f-170434f63b05-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "904747b9-652d-4e0f-8d8f-170434f63b05" (UID: "904747b9-652d-4e0f-8d8f-170434f63b05"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:21:02.461991 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.461966 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/904747b9-652d-4e0f-8d8f-170434f63b05-kube-api-access-p7rvf" (OuterVolumeSpecName: "kube-api-access-p7rvf") pod "904747b9-652d-4e0f-8d8f-170434f63b05" (UID: "904747b9-652d-4e0f-8d8f-170434f63b05"). InnerVolumeSpecName "kube-api-access-p7rvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:21:02.560580 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.560556 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/904747b9-652d-4e0f-8d8f-170434f63b05-extensions-socket-volume\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:21:02.560580 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:02.560577 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7rvf\" (UniqueName: \"kubernetes.io/projected/904747b9-652d-4e0f-8d8f-170434f63b05-kube-api-access-p7rvf\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:21:03.174029 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.173991 2585 generic.go:356] "Generic (PLEG): container finished" podID="904747b9-652d-4e0f-8d8f-170434f63b05" containerID="a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647" exitCode=0 Sep 19 09:21:03.174426 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.174037 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" Sep 19 09:21:03.174426 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.174057 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" event={"ID":"904747b9-652d-4e0f-8d8f-170434f63b05","Type":"ContainerDied","Data":"a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647"} Sep 19 09:21:03.174426 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.174099 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf" event={"ID":"904747b9-652d-4e0f-8d8f-170434f63b05","Type":"ContainerDied","Data":"08e246bb36abb49b4e470dfb118aa0f9a0c3fc94ee1461991cb76dcdb39ce07b"} Sep 19 09:21:03.174426 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.174119 2585 scope.go:117] "RemoveContainer" containerID="a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647" Sep 19 09:21:03.183222 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.183209 2585 scope.go:117] "RemoveContainer" containerID="a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647" Sep 19 09:21:03.183507 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:21:03.183487 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647\": container with ID starting with a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647 not found: ID does not exist" containerID="a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647" Sep 19 09:21:03.183664 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.183541 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647"} err="failed to get container status \"a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647\": rpc error: code = NotFound desc = could not find container \"a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647\": container with ID starting with a7be3360766645d84ae53cfc6644593687898598e07259d75af37a70a3662647 not found: ID does not exist" Sep 19 09:21:03.196930 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.196909 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf"] Sep 19 09:21:03.203717 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.203694 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-tp8wf"] Sep 19 09:21:03.435257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:03.435179 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="904747b9-652d-4e0f-8d8f-170434f63b05" path="/var/lib/kubelet/pods/904747b9-652d-4e0f-8d8f-170434f63b05/volumes" Sep 19 09:21:21.877368 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.877332 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r2925"] Sep 19 09:21:21.877800 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.877705 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="904747b9-652d-4e0f-8d8f-170434f63b05" containerName="manager" Sep 19 09:21:21.877800 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.877717 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="904747b9-652d-4e0f-8d8f-170434f63b05" containerName="manager" Sep 19 09:21:21.877800 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.877784 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="904747b9-652d-4e0f-8d8f-170434f63b05" containerName="manager" Sep 19 09:21:21.880719 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.880701 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:21.883151 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.883127 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 19 09:21:21.883262 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.883150 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kdmd6\"" Sep 19 09:21:21.891207 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.891183 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r2925"] Sep 19 09:21:21.973758 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:21.973723 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r2925"] Sep 19 09:21:22.018046 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.018013 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-config-file\") pod \"limitador-limitador-8588749977-r2925\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:22.018046 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.018046 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brjxz\" (UniqueName: \"kubernetes.io/projected/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-kube-api-access-brjxz\") pod \"limitador-limitador-8588749977-r2925\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:22.118956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.118921 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-config-file\") pod \"limitador-limitador-8588749977-r2925\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:22.118956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.118954 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brjxz\" (UniqueName: \"kubernetes.io/projected/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-kube-api-access-brjxz\") pod \"limitador-limitador-8588749977-r2925\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:22.119647 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.119621 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-config-file\") pod \"limitador-limitador-8588749977-r2925\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:22.127261 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.127239 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brjxz\" (UniqueName: \"kubernetes.io/projected/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-kube-api-access-brjxz\") pod \"limitador-limitador-8588749977-r2925\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:22.191816 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.191734 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:22.465460 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.465436 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r2925"] Sep 19 09:21:22.467938 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:21:22.467911 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod471ea19b_1350_44a3_96d9_a7ef0b3b75f7.slice/crio-b1dd58153da9487ae5a337f60b069aaa5beaa1da5b6e824db5445a161af0cddc WatchSource:0}: Error finding container b1dd58153da9487ae5a337f60b069aaa5beaa1da5b6e824db5445a161af0cddc: Status 404 returned error can't find the container with id b1dd58153da9487ae5a337f60b069aaa5beaa1da5b6e824db5445a161af0cddc Sep 19 09:21:22.792534 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.792445 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-68str"] Sep 19 09:21:22.797043 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.797022 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-68str" Sep 19 09:21:22.799556 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.799538 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qs5ww\"" Sep 19 09:21:22.805212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.805164 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-68str"] Sep 19 09:21:22.926605 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:22.926555 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gzch\" (UniqueName: \"kubernetes.io/projected/8c4294cf-cf8c-4c8d-bd58-e1de1b12c484-kube-api-access-8gzch\") pod \"authorino-59c974bd48-68str\" (UID: \"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484\") " pod="kuadrant-system/authorino-59c974bd48-68str" Sep 19 09:21:23.028022 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:23.027982 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gzch\" (UniqueName: \"kubernetes.io/projected/8c4294cf-cf8c-4c8d-bd58-e1de1b12c484-kube-api-access-8gzch\") pod \"authorino-59c974bd48-68str\" (UID: \"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484\") " pod="kuadrant-system/authorino-59c974bd48-68str" Sep 19 09:21:23.036138 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:23.036105 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gzch\" (UniqueName: \"kubernetes.io/projected/8c4294cf-cf8c-4c8d-bd58-e1de1b12c484-kube-api-access-8gzch\") pod \"authorino-59c974bd48-68str\" (UID: \"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484\") " pod="kuadrant-system/authorino-59c974bd48-68str" Sep 19 09:21:23.107058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:23.107022 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-68str" Sep 19 09:21:23.283203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:23.283161 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r2925" event={"ID":"471ea19b-1350-44a3-96d9-a7ef0b3b75f7","Type":"ContainerStarted","Data":"b1dd58153da9487ae5a337f60b069aaa5beaa1da5b6e824db5445a161af0cddc"} Sep 19 09:21:23.378932 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:23.378844 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-68str"] Sep 19 09:21:23.496774 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:21:23.496734 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c4294cf_cf8c_4c8d_bd58_e1de1b12c484.slice/crio-8c6fe8e8b400c708d26e9efe7bd8fee2c479fd2f8fd60ed44ee55881e9f417a8 WatchSource:0}: Error finding container 8c6fe8e8b400c708d26e9efe7bd8fee2c479fd2f8fd60ed44ee55881e9f417a8: Status 404 returned error can't find the container with id 8c6fe8e8b400c708d26e9efe7bd8fee2c479fd2f8fd60ed44ee55881e9f417a8 Sep 19 09:21:24.292203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:24.292170 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r2925" event={"ID":"471ea19b-1350-44a3-96d9-a7ef0b3b75f7","Type":"ContainerStarted","Data":"04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53"} Sep 19 09:21:24.292693 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:24.292293 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:24.293531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:24.293499 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-68str" event={"ID":"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484","Type":"ContainerStarted","Data":"8c6fe8e8b400c708d26e9efe7bd8fee2c479fd2f8fd60ed44ee55881e9f417a8"} Sep 19 09:21:24.307338 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:24.307295 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-r2925" podStartSLOduration=2.225035255 podStartE2EDuration="3.307282731s" totalImagesPullingTime="1.082247476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:21:24.306163491 +0000 UTC m=+891.064284515" watchObservedRunningTime="2026-09-19 09:21:24.307282731 +0000 UTC m=+891.065403755" Sep 19 09:21:26.306003 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:26.305967 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-68str" event={"ID":"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484","Type":"ContainerStarted","Data":"b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261"} Sep 19 09:21:26.320733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:26.320687 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-68str" podStartSLOduration=2.357197183 podStartE2EDuration="4.320673129s" totalImagesPullingTime="1.963475946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:21:26.319987219 +0000 UTC m=+893.078108235" watchObservedRunningTime="2026-09-19 09:21:26.320673129 +0000 UTC m=+893.078794153" Sep 19 09:21:35.300209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:35.300176 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:35.436303 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:35.436270 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:21:35.436874 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:35.436851 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:21:35.440154 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:35.440135 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:21:35.440789 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:35.440773 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:21:36.385469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:36.385436 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r2925"] Sep 19 09:21:36.385866 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:36.385628 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-r2925" podUID="471ea19b-1350-44a3-96d9-a7ef0b3b75f7" containerName="limitador" containerID="cri-o://04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53" gracePeriod=30 Sep 19 09:21:36.932779 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:36.932755 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:37.054939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.054915 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brjxz\" (UniqueName: \"kubernetes.io/projected/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-kube-api-access-brjxz\") pod \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " Sep 19 09:21:37.055061 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.054961 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-config-file\") pod \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\" (UID: \"471ea19b-1350-44a3-96d9-a7ef0b3b75f7\") " Sep 19 09:21:37.055319 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.055296 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-config-file" (OuterVolumeSpecName: "config-file") pod "471ea19b-1350-44a3-96d9-a7ef0b3b75f7" (UID: "471ea19b-1350-44a3-96d9-a7ef0b3b75f7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:21:37.056869 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.056850 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-kube-api-access-brjxz" (OuterVolumeSpecName: "kube-api-access-brjxz") pod "471ea19b-1350-44a3-96d9-a7ef0b3b75f7" (UID: "471ea19b-1350-44a3-96d9-a7ef0b3b75f7"). InnerVolumeSpecName "kube-api-access-brjxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:21:37.156400 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.156360 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brjxz\" (UniqueName: \"kubernetes.io/projected/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-kube-api-access-brjxz\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:21:37.156400 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.156397 2585 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/471ea19b-1350-44a3-96d9-a7ef0b3b75f7-config-file\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:21:37.363425 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.363349 2585 generic.go:356] "Generic (PLEG): container finished" podID="471ea19b-1350-44a3-96d9-a7ef0b3b75f7" containerID="04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53" exitCode=0 Sep 19 09:21:37.363425 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.363392 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r2925" Sep 19 09:21:37.363624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.363434 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r2925" event={"ID":"471ea19b-1350-44a3-96d9-a7ef0b3b75f7","Type":"ContainerDied","Data":"04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53"} Sep 19 09:21:37.363624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.363490 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r2925" event={"ID":"471ea19b-1350-44a3-96d9-a7ef0b3b75f7","Type":"ContainerDied","Data":"b1dd58153da9487ae5a337f60b069aaa5beaa1da5b6e824db5445a161af0cddc"} Sep 19 09:21:37.363624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.363543 2585 scope.go:117] "RemoveContainer" containerID="04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53" Sep 19 09:21:37.371872 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.371855 2585 scope.go:117] "RemoveContainer" containerID="04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53" Sep 19 09:21:37.372118 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:21:37.372099 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53\": container with ID starting with 04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53 not found: ID does not exist" containerID="04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53" Sep 19 09:21:37.372181 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.372134 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53"} err="failed to get container status \"04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53\": rpc error: code = NotFound desc = could not find container \"04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53\": container with ID starting with 04a4bffc27a8c476f9b3afffa546304f2856e94305c575557549e121d1ad0e53 not found: ID does not exist" Sep 19 09:21:37.384119 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.384088 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r2925"] Sep 19 09:21:37.387584 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.387561 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r2925"] Sep 19 09:21:37.436275 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:37.436243 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="471ea19b-1350-44a3-96d9-a7ef0b3b75f7" path="/var/lib/kubelet/pods/471ea19b-1350-44a3-96d9-a7ef0b3b75f7/volumes" Sep 19 09:21:39.321590 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.321556 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9"] Sep 19 09:21:39.322067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.322050 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="471ea19b-1350-44a3-96d9-a7ef0b3b75f7" containerName="limitador" Sep 19 09:21:39.322148 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.322071 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="471ea19b-1350-44a3-96d9-a7ef0b3b75f7" containerName="limitador" Sep 19 09:21:39.322200 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.322158 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="471ea19b-1350-44a3-96d9-a7ef0b3b75f7" containerName="limitador" Sep 19 09:21:39.326306 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.326287 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.328989 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.328963 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 19 09:21:39.329100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.328967 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-777gx\"" Sep 19 09:21:39.329100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.329031 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 19 09:21:39.329800 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.329783 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 19 09:21:39.338293 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.338274 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9"] Sep 19 09:21:39.473753 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.473718 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnrf4\" (UniqueName: \"kubernetes.io/projected/780c2ea2-5d2a-4438-9dae-77b6e12dbe45-kube-api-access-hnrf4\") pod \"postgres-69fb4594cf-pcnv9\" (UID: \"780c2ea2-5d2a-4438-9dae-77b6e12dbe45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.473938 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.473813 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/780c2ea2-5d2a-4438-9dae-77b6e12dbe45-data\") pod \"postgres-69fb4594cf-pcnv9\" (UID: \"780c2ea2-5d2a-4438-9dae-77b6e12dbe45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.575186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.575109 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnrf4\" (UniqueName: \"kubernetes.io/projected/780c2ea2-5d2a-4438-9dae-77b6e12dbe45-kube-api-access-hnrf4\") pod \"postgres-69fb4594cf-pcnv9\" (UID: \"780c2ea2-5d2a-4438-9dae-77b6e12dbe45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.575186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.575149 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/780c2ea2-5d2a-4438-9dae-77b6e12dbe45-data\") pod \"postgres-69fb4594cf-pcnv9\" (UID: \"780c2ea2-5d2a-4438-9dae-77b6e12dbe45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.575498 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.575482 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/780c2ea2-5d2a-4438-9dae-77b6e12dbe45-data\") pod \"postgres-69fb4594cf-pcnv9\" (UID: \"780c2ea2-5d2a-4438-9dae-77b6e12dbe45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.583984 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.583955 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnrf4\" (UniqueName: \"kubernetes.io/projected/780c2ea2-5d2a-4438-9dae-77b6e12dbe45-kube-api-access-hnrf4\") pod \"postgres-69fb4594cf-pcnv9\" (UID: \"780c2ea2-5d2a-4438-9dae-77b6e12dbe45\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.636793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.636763 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:39.905484 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:39.905462 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9"] Sep 19 09:21:39.907597 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:21:39.907561 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780c2ea2_5d2a_4438_9dae_77b6e12dbe45.slice/crio-8a0272108682f349a2de9b461f16bff1daa65551675af7bba4cb2dab3eca7664 WatchSource:0}: Error finding container 8a0272108682f349a2de9b461f16bff1daa65551675af7bba4cb2dab3eca7664: Status 404 returned error can't find the container with id 8a0272108682f349a2de9b461f16bff1daa65551675af7bba4cb2dab3eca7664 Sep 19 09:21:40.379766 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:40.379730 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" event={"ID":"780c2ea2-5d2a-4438-9dae-77b6e12dbe45","Type":"ContainerStarted","Data":"8a0272108682f349a2de9b461f16bff1daa65551675af7bba4cb2dab3eca7664"} Sep 19 09:21:45.415989 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:45.415939 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" event={"ID":"780c2ea2-5d2a-4438-9dae-77b6e12dbe45","Type":"ContainerStarted","Data":"87fe92e17883fbcab2a8845ecef5cc4b7642e58740dca3c56023cc1d3c6f711e"} Sep 19 09:21:45.416480 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:45.416020 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:45.432989 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:45.432937 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" podStartSLOduration=1.802918799 podStartE2EDuration="6.432922034s" totalImagesPullingTime="4.630003235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:21:45.431658104 +0000 UTC m=+912.189779130" watchObservedRunningTime="2026-09-19 09:21:45.432922034 +0000 UTC m=+912.191043037" Sep 19 09:21:51.449636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:51.449598 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-pcnv9" Sep 19 09:21:52.150579 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.150543 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-7tpr4"] Sep 19 09:21:52.153912 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.153889 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.156174 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.156156 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 19 09:21:52.161068 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.161046 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-7tpr4"] Sep 19 09:21:52.184067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.184033 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a08222e-eb11-4f82-9859-0d83de6371a8-tls-cert\") pod \"authorino-6d4dfc649f-7tpr4\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.184214 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.184085 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbzkc\" (UniqueName: \"kubernetes.io/projected/1a08222e-eb11-4f82-9859-0d83de6371a8-kube-api-access-mbzkc\") pod \"authorino-6d4dfc649f-7tpr4\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.229246 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.229210 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-7tpr4"] Sep 19 09:21:52.229497 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:21:52.229475 2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mbzkc tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" podUID="1a08222e-eb11-4f82-9859-0d83de6371a8" Sep 19 09:21:52.263737 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.263701 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55966b5b48-ls2hw"] Sep 19 09:21:52.266886 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.266865 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.276797 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.276773 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55966b5b48-ls2hw"] Sep 19 09:21:52.284681 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.284656 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1625a8bd-9b29-44a9-8b9d-f9330f00f962-tls-cert\") pod \"authorino-55966b5b48-ls2hw\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.284811 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.284685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57qnx\" (UniqueName: \"kubernetes.io/projected/1625a8bd-9b29-44a9-8b9d-f9330f00f962-kube-api-access-57qnx\") pod \"authorino-55966b5b48-ls2hw\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.284811 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.284709 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a08222e-eb11-4f82-9859-0d83de6371a8-tls-cert\") pod \"authorino-6d4dfc649f-7tpr4\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.284811 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.284775 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbzkc\" (UniqueName: \"kubernetes.io/projected/1a08222e-eb11-4f82-9859-0d83de6371a8-kube-api-access-mbzkc\") pod \"authorino-6d4dfc649f-7tpr4\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.287412 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.287387 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a08222e-eb11-4f82-9859-0d83de6371a8-tls-cert\") pod \"authorino-6d4dfc649f-7tpr4\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.295273 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.295244 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbzkc\" (UniqueName: \"kubernetes.io/projected/1a08222e-eb11-4f82-9859-0d83de6371a8-kube-api-access-mbzkc\") pod \"authorino-6d4dfc649f-7tpr4\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.386143 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.386103 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1625a8bd-9b29-44a9-8b9d-f9330f00f962-tls-cert\") pod \"authorino-55966b5b48-ls2hw\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.386143 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.386144 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57qnx\" (UniqueName: \"kubernetes.io/projected/1625a8bd-9b29-44a9-8b9d-f9330f00f962-kube-api-access-57qnx\") pod \"authorino-55966b5b48-ls2hw\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.388779 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.388749 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1625a8bd-9b29-44a9-8b9d-f9330f00f962-tls-cert\") pod \"authorino-55966b5b48-ls2hw\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.395491 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.395454 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57qnx\" (UniqueName: \"kubernetes.io/projected/1625a8bd-9b29-44a9-8b9d-f9330f00f962-kube-api-access-57qnx\") pod \"authorino-55966b5b48-ls2hw\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.450602 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.450529 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.460632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.460609 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:52.486789 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.486758 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbzkc\" (UniqueName: \"kubernetes.io/projected/1a08222e-eb11-4f82-9859-0d83de6371a8-kube-api-access-mbzkc\") pod \"1a08222e-eb11-4f82-9859-0d83de6371a8\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " Sep 19 09:21:52.486936 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.486797 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a08222e-eb11-4f82-9859-0d83de6371a8-tls-cert\") pod \"1a08222e-eb11-4f82-9859-0d83de6371a8\" (UID: \"1a08222e-eb11-4f82-9859-0d83de6371a8\") " Sep 19 09:21:52.488964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.488924 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a08222e-eb11-4f82-9859-0d83de6371a8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1a08222e-eb11-4f82-9859-0d83de6371a8" (UID: "1a08222e-eb11-4f82-9859-0d83de6371a8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:21:52.488964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.488954 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a08222e-eb11-4f82-9859-0d83de6371a8-kube-api-access-mbzkc" (OuterVolumeSpecName: "kube-api-access-mbzkc") pod "1a08222e-eb11-4f82-9859-0d83de6371a8" (UID: "1a08222e-eb11-4f82-9859-0d83de6371a8"). InnerVolumeSpecName "kube-api-access-mbzkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:21:52.576506 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.576474 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:21:52.587611 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.587589 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbzkc\" (UniqueName: \"kubernetes.io/projected/1a08222e-eb11-4f82-9859-0d83de6371a8-kube-api-access-mbzkc\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:21:52.587707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.587614 2585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1a08222e-eb11-4f82-9859-0d83de6371a8-tls-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:21:52.847978 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:52.847952 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55966b5b48-ls2hw"] Sep 19 09:21:52.849825 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:21:52.849799 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1625a8bd_9b29_44a9_8b9d_f9330f00f962.slice/crio-1352af096cbff10a1373ee04e79af2d73aebfa7675e1d8f59b426a5d98c90060 WatchSource:0}: Error finding container 1352af096cbff10a1373ee04e79af2d73aebfa7675e1d8f59b426a5d98c90060: Status 404 returned error can't find the container with id 1352af096cbff10a1373ee04e79af2d73aebfa7675e1d8f59b426a5d98c90060 Sep 19 09:21:53.457734 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.457700 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55966b5b48-ls2hw" event={"ID":"1625a8bd-9b29-44a9-8b9d-f9330f00f962","Type":"ContainerStarted","Data":"f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0"} Sep 19 09:21:53.458120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.457747 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55966b5b48-ls2hw" event={"ID":"1625a8bd-9b29-44a9-8b9d-f9330f00f962","Type":"ContainerStarted","Data":"1352af096cbff10a1373ee04e79af2d73aebfa7675e1d8f59b426a5d98c90060"} Sep 19 09:21:53.458120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.457770 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-7tpr4" Sep 19 09:21:53.474835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.474789 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55966b5b48-ls2hw" podStartSLOduration=0.968618921 podStartE2EDuration="1.474771804s" totalImagesPullingTime="506.152883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:21:53.472438634 +0000 UTC m=+920.230559660" watchObservedRunningTime="2026-09-19 09:21:53.474771804 +0000 UTC m=+920.232892829" Sep 19 09:21:53.503986 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.503918 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-7tpr4"] Sep 19 09:21:53.512602 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.512567 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-68str"] Sep 19 09:21:53.512878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.512835 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-68str" podUID="8c4294cf-cf8c-4c8d-bd58-e1de1b12c484" containerName="authorino" containerID="cri-o://b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261" gracePeriod=30 Sep 19 09:21:53.516112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.515910 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-7tpr4"] Sep 19 09:21:53.746412 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.746387 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-68str" Sep 19 09:21:53.797165 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.797135 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gzch\" (UniqueName: \"kubernetes.io/projected/8c4294cf-cf8c-4c8d-bd58-e1de1b12c484-kube-api-access-8gzch\") pod \"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484\" (UID: \"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484\") " Sep 19 09:21:53.799169 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.799138 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c4294cf-cf8c-4c8d-bd58-e1de1b12c484-kube-api-access-8gzch" (OuterVolumeSpecName: "kube-api-access-8gzch") pod "8c4294cf-cf8c-4c8d-bd58-e1de1b12c484" (UID: "8c4294cf-cf8c-4c8d-bd58-e1de1b12c484"). InnerVolumeSpecName "kube-api-access-8gzch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:21:53.898108 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:53.898039 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gzch\" (UniqueName: \"kubernetes.io/projected/8c4294cf-cf8c-4c8d-bd58-e1de1b12c484-kube-api-access-8gzch\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:21:54.463689 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.463653 2585 generic.go:356] "Generic (PLEG): container finished" podID="8c4294cf-cf8c-4c8d-bd58-e1de1b12c484" containerID="b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261" exitCode=0 Sep 19 09:21:54.464091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.463694 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-68str" Sep 19 09:21:54.464091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.463729 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-68str" event={"ID":"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484","Type":"ContainerDied","Data":"b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261"} Sep 19 09:21:54.464091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.463764 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-68str" event={"ID":"8c4294cf-cf8c-4c8d-bd58-e1de1b12c484","Type":"ContainerDied","Data":"8c6fe8e8b400c708d26e9efe7bd8fee2c479fd2f8fd60ed44ee55881e9f417a8"} Sep 19 09:21:54.464091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.463781 2585 scope.go:117] "RemoveContainer" containerID="b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261" Sep 19 09:21:54.473127 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.473109 2585 scope.go:117] "RemoveContainer" containerID="b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261" Sep 19 09:21:54.473372 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:21:54.473353 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261\": container with ID starting with b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261 not found: ID does not exist" containerID="b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261" Sep 19 09:21:54.473440 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.473386 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261"} err="failed to get container status \"b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261\": rpc error: code = NotFound desc = could not find container \"b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261\": container with ID starting with b86ec1c6a26fa1a1739db6d0bec4fde55f9288b0b61e6bb7a8b94001ae97d261 not found: ID does not exist" Sep 19 09:21:54.484404 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.484380 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-68str"] Sep 19 09:21:54.488420 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:54.488400 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-68str"] Sep 19 09:21:55.436186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:55.436151 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a08222e-eb11-4f82-9859-0d83de6371a8" path="/var/lib/kubelet/pods/1a08222e-eb11-4f82-9859-0d83de6371a8/volumes" Sep 19 09:21:55.436452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:55.436435 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c4294cf-cf8c-4c8d-bd58-e1de1b12c484" path="/var/lib/kubelet/pods/8c4294cf-cf8c-4c8d-bd58-e1de1b12c484/volumes" Sep 19 09:21:58.253322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.253290 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59b595cf4b-sqbbj"] Sep 19 09:21:58.254189 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.253682 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c4294cf-cf8c-4c8d-bd58-e1de1b12c484" containerName="authorino" Sep 19 09:21:58.254189 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.253694 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c4294cf-cf8c-4c8d-bd58-e1de1b12c484" containerName="authorino" Sep 19 09:21:58.254189 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.253748 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c4294cf-cf8c-4c8d-bd58-e1de1b12c484" containerName="authorino" Sep 19 09:21:58.277148 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.277117 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b595cf4b-sqbbj"] Sep 19 09:21:58.277326 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.277241 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.279852 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.279811 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 19 09:21:58.280004 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.279964 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 19 09:21:58.280830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.280809 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 19 09:21:58.280830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.280805 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9txm5\"" Sep 19 09:21:58.334355 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.334323 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hcbb\" (UniqueName: \"kubernetes.io/projected/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-kube-api-access-8hcbb\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.334497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.334394 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-webhook-cert\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.334497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.334454 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.435104 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.435065 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5fbf54c957-fh89x"] Sep 19 09:21:58.435874 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.435835 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-webhook-cert\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.436885 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:21:58.436851 2585 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 19 09:21:58.437694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.435936 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.437694 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:21:58.437069 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs podName:5c3ee14d-9a6a-4fb6-bffd-b69345ea5068 nodeName:}" failed. No retries permitted until 2026-09-19 09:21:58.936912301 +0000 UTC m=+925.695033321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs") pod "maas-controller-59b595cf4b-sqbbj" (UID: "5c3ee14d-9a6a-4fb6-bffd-b69345ea5068") : secret "maas-controller-metrics-tls" not found Sep 19 09:21:58.437694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.437132 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hcbb\" (UniqueName: \"kubernetes.io/projected/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-kube-api-access-8hcbb\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.442894 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.442869 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.448292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.448267 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-webhook-cert\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.450761 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.450740 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-fh89x"] Sep 19 09:21:58.454207 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.454188 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hcbb\" (UniqueName: \"kubernetes.io/projected/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-kube-api-access-8hcbb\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.538282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.538197 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46hxv\" (UniqueName: \"kubernetes.io/projected/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-kube-api-access-46hxv\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.538282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.538247 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-webhook-cert\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.538282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.538282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-metrics-certs\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.638967 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.638931 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46hxv\" (UniqueName: \"kubernetes.io/projected/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-kube-api-access-46hxv\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.638967 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.638968 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-webhook-cert\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.639167 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.638993 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-metrics-certs\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.641365 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.641336 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-webhook-cert\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.641479 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.641382 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-metrics-certs\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.646945 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.646927 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46hxv\" (UniqueName: \"kubernetes.io/projected/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-kube-api-access-46hxv\") pod \"maas-controller-5fbf54c957-fh89x\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.771620 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.771581 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:21:58.942537 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.942484 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:58.944817 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:58.944798 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs\") pod \"maas-controller-59b595cf4b-sqbbj\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:59.040100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:59.040072 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-fh89x"] Sep 19 09:21:59.041556 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:21:59.041509 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefed2e43_a0a7_41c9_a0ff_1def3d7677bb.slice/crio-2fa192126fa52766bac70c5e4a0814b03903276e4fbb457474e750f67d7effc9 WatchSource:0}: Error finding container 2fa192126fa52766bac70c5e4a0814b03903276e4fbb457474e750f67d7effc9: Status 404 returned error can't find the container with id 2fa192126fa52766bac70c5e4a0814b03903276e4fbb457474e750f67d7effc9 Sep 19 09:21:59.189224 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:59.189134 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:21:59.460383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:59.460359 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b595cf4b-sqbbj"] Sep 19 09:21:59.462564 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:21:59.462536 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c3ee14d_9a6a_4fb6_bffd_b69345ea5068.slice/crio-e8cd38396b84f4cbd9cf7c7b00fdfcb6d6658d061de5429bfecde08d1e5fcb48 WatchSource:0}: Error finding container e8cd38396b84f4cbd9cf7c7b00fdfcb6d6658d061de5429bfecde08d1e5fcb48: Status 404 returned error can't find the container with id e8cd38396b84f4cbd9cf7c7b00fdfcb6d6658d061de5429bfecde08d1e5fcb48 Sep 19 09:21:59.490902 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:59.490873 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" event={"ID":"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068","Type":"ContainerStarted","Data":"e8cd38396b84f4cbd9cf7c7b00fdfcb6d6658d061de5429bfecde08d1e5fcb48"} Sep 19 09:21:59.492067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:21:59.492043 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-fh89x" event={"ID":"efed2e43-a0a7-41c9-a0ff-1def3d7677bb","Type":"ContainerStarted","Data":"2fa192126fa52766bac70c5e4a0814b03903276e4fbb457474e750f67d7effc9"} Sep 19 09:22:02.513690 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:02.513654 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-fh89x" event={"ID":"efed2e43-a0a7-41c9-a0ff-1def3d7677bb","Type":"ContainerStarted","Data":"96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170"} Sep 19 09:22:02.514123 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:02.513745 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:22:02.515684 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:02.515661 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" event={"ID":"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068","Type":"ContainerStarted","Data":"5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b"} Sep 19 09:22:02.515814 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:02.515707 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:22:02.538285 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:02.538241 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5fbf54c957-fh89x" podStartSLOduration=2.034411311 podStartE2EDuration="4.538228015s" totalImagesPullingTime="2.503816704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:22:02.537306467 +0000 UTC m=+929.295427493" watchObservedRunningTime="2026-09-19 09:22:02.538228015 +0000 UTC m=+929.296349040" Sep 19 09:22:02.558977 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:02.558925 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" podStartSLOduration=2.466397451 podStartE2EDuration="4.558910951s" totalImagesPullingTime="2.0925135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:22:02.557410287 +0000 UTC m=+929.315531313" watchObservedRunningTime="2026-09-19 09:22:02.558910951 +0000 UTC m=+929.317031975" Sep 19 09:22:13.523251 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.523218 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:22:13.523772 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.523491 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:22:13.590727 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.590694 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b595cf4b-sqbbj"] Sep 19 09:22:13.590963 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.590933 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" podUID="5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" containerName="manager" containerID="cri-o://5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b" gracePeriod=30 Sep 19 09:22:13.827452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.827431 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:22:13.968841 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.968813 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs\") pod \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " Sep 19 09:22:13.969017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.968914 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hcbb\" (UniqueName: \"kubernetes.io/projected/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-kube-api-access-8hcbb\") pod \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " Sep 19 09:22:13.969017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.968952 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-webhook-cert\") pod \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\" (UID: \"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068\") " Sep 19 09:22:13.970992 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.970964 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" (UID: "5c3ee14d-9a6a-4fb6-bffd-b69345ea5068"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:22:13.970992 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.970978 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" (UID: "5c3ee14d-9a6a-4fb6-bffd-b69345ea5068"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:22:13.971127 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:13.971061 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-kube-api-access-8hcbb" (OuterVolumeSpecName: "kube-api-access-8hcbb") pod "5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" (UID: "5c3ee14d-9a6a-4fb6-bffd-b69345ea5068"). InnerVolumeSpecName "kube-api-access-8hcbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:22:14.070323 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.070244 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hcbb\" (UniqueName: \"kubernetes.io/projected/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-kube-api-access-8hcbb\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:22:14.070323 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.070271 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-webhook-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:22:14.070323 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.070279 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:22:14.583123 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.583090 2585 generic.go:356] "Generic (PLEG): container finished" podID="5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" containerID="5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b" exitCode=0 Sep 19 09:22:14.583123 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.583126 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" Sep 19 09:22:14.583592 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.583163 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" event={"ID":"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068","Type":"ContainerDied","Data":"5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b"} Sep 19 09:22:14.583592 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.583210 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b595cf4b-sqbbj" event={"ID":"5c3ee14d-9a6a-4fb6-bffd-b69345ea5068","Type":"ContainerDied","Data":"e8cd38396b84f4cbd9cf7c7b00fdfcb6d6658d061de5429bfecde08d1e5fcb48"} Sep 19 09:22:14.583592 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.583235 2585 scope.go:117] "RemoveContainer" containerID="5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b" Sep 19 09:22:14.592320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.592151 2585 scope.go:117] "RemoveContainer" containerID="5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b" Sep 19 09:22:14.592427 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:22:14.592409 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b\": container with ID starting with 5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b not found: ID does not exist" containerID="5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b" Sep 19 09:22:14.592473 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.592437 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b"} err="failed to get container status \"5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b\": rpc error: code = NotFound desc = could not find container \"5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b\": container with ID starting with 5947ecb1e9bf4bd003ff35a015c2812cfe4616a45a68f59a12239b7599aab92b not found: ID does not exist" Sep 19 09:22:14.604754 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.604727 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b595cf4b-sqbbj"] Sep 19 09:22:14.608491 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:14.608471 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59b595cf4b-sqbbj"] Sep 19 09:22:15.436423 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:15.436391 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" path="/var/lib/kubelet/pods/5c3ee14d-9a6a-4fb6-bffd-b69345ea5068/volumes" Sep 19 09:22:34.145761 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.145707 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-fqhvv"] Sep 19 09:22:34.149630 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.146259 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" containerName="manager" Sep 19 09:22:34.149630 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.146279 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" containerName="manager" Sep 19 09:22:34.149630 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.146362 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c3ee14d-9a6a-4fb6-bffd-b69345ea5068" containerName="manager" Sep 19 09:22:34.149835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.149638 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.152305 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.152284 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 19 09:22:34.152451 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.152310 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 19 09:22:34.152936 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.152918 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzltl\"" Sep 19 09:22:34.167019 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.166988 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-fqhvv"] Sep 19 09:22:34.237496 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.237464 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2372b3-d80a-4037-9743-9453b42e9299-openshift-service-ca\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.237496 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.237499 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.237782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.237534 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-maas-api-tls\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.237782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.237664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hvq2\" (UniqueName: \"kubernetes.io/projected/4c2372b3-d80a-4037-9743-9453b42e9299-kube-api-access-6hvq2\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.338906 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.338866 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hvq2\" (UniqueName: \"kubernetes.io/projected/4c2372b3-d80a-4037-9743-9453b42e9299-kube-api-access-6hvq2\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.339096 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.338966 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2372b3-d80a-4037-9743-9453b42e9299-openshift-service-ca\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.339096 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.338991 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.339096 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.339024 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-maas-api-tls\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.339264 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:22:34.339151 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 19 09:22:34.339264 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:22:34.339250 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs podName:4c2372b3-d80a-4037-9743-9453b42e9299 nodeName:}" failed. No retries permitted until 2026-09-19 09:22:34.839228767 +0000 UTC m=+961.597349792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs") pod "maas-api-75876448df-fqhvv" (UID: "4c2372b3-d80a-4037-9743-9453b42e9299") : secret "maas-api-metrics-tls" not found Sep 19 09:22:34.339669 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.339644 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2372b3-d80a-4037-9743-9453b42e9299-openshift-service-ca\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.341472 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.341449 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-maas-api-tls\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.351467 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.351440 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hvq2\" (UniqueName: \"kubernetes.io/projected/4c2372b3-d80a-4037-9743-9453b42e9299-kube-api-access-6hvq2\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.845155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.845115 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:34.847727 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:34.847699 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs\") pod \"maas-api-75876448df-fqhvv\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:35.061340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:35.061304 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:35.338792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:35.338768 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-fqhvv"] Sep 19 09:22:35.341144 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:22:35.341107 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2372b3_d80a_4037_9743_9453b42e9299.slice/crio-e7afbe608e85c96ab742c67104e0f53e7dd5fd647c1d0c2fe046857a9ef9515c WatchSource:0}: Error finding container e7afbe608e85c96ab742c67104e0f53e7dd5fd647c1d0c2fe046857a9ef9515c: Status 404 returned error can't find the container with id e7afbe608e85c96ab742c67104e0f53e7dd5fd647c1d0c2fe046857a9ef9515c Sep 19 09:22:35.342474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:35.342451 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:22:35.690242 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:35.690200 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" event={"ID":"4c2372b3-d80a-4037-9743-9453b42e9299","Type":"ContainerStarted","Data":"e7afbe608e85c96ab742c67104e0f53e7dd5fd647c1d0c2fe046857a9ef9515c"} Sep 19 09:22:37.703165 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:37.703126 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" event={"ID":"4c2372b3-d80a-4037-9743-9453b42e9299","Type":"ContainerStarted","Data":"5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a"} Sep 19 09:22:37.703564 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:37.703277 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:22:37.728757 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:37.728712 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" podStartSLOduration=2.080925611 podStartE2EDuration="3.728698026s" totalImagesPullingTime="1.647772415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:22:37.726640108 +0000 UTC m=+964.484761133" watchObservedRunningTime="2026-09-19 09:22:37.728698026 +0000 UTC m=+964.486819050" Sep 19 09:22:43.713055 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:22:43.713020 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:23:02.374075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.373997 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75"] Sep 19 09:23:02.377664 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.377644 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.381482 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.381452 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-942q6\"" Sep 19 09:23:02.381597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.381501 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 19 09:23:02.381597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.381461 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Sep 19 09:23:02.381736 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.381664 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 19 09:23:02.387738 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.387715 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75"] Sep 19 09:23:02.449778 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.449747 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.449959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.449786 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eedad42a-3003-4b50-b4e9-0137f683f144-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.449959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.449855 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.449959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.449913 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.450107 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.449957 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh49r\" (UniqueName: \"kubernetes.io/projected/eedad42a-3003-4b50-b4e9-0137f683f144-kube-api-access-mh49r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.450107 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.450007 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.551277 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551241 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.551462 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551439 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.551560 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551495 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eedad42a-3003-4b50-b4e9-0137f683f144-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.551632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551569 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.551632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551616 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.551749 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551655 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh49r\" (UniqueName: \"kubernetes.io/projected/eedad42a-3003-4b50-b4e9-0137f683f144-kube-api-access-mh49r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.551830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551812 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.552012 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.551986 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.552131 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.552112 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.553842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.553822 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eedad42a-3003-4b50-b4e9-0137f683f144-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.554389 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.554373 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eedad42a-3003-4b50-b4e9-0137f683f144-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.561852 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.561830 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh49r\" (UniqueName: \"kubernetes.io/projected/eedad42a-3003-4b50-b4e9-0137f683f144-kube-api-access-mh49r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75\" (UID: \"eedad42a-3003-4b50-b4e9-0137f683f144\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.688854 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.688796 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:02.996761 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:02.996731 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75"] Sep 19 09:23:03.000396 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:23:03.000358 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeedad42a_3003_4b50_b4e9_0137f683f144.slice/crio-8bf689f814e2fd94aa53ea73228ff98a7bf9424c85271b3c22164fe795a04373 WatchSource:0}: Error finding container 8bf689f814e2fd94aa53ea73228ff98a7bf9424c85271b3c22164fe795a04373: Status 404 returned error can't find the container with id 8bf689f814e2fd94aa53ea73228ff98a7bf9424c85271b3c22164fe795a04373 Sep 19 09:23:03.848491 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:03.848446 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" event={"ID":"eedad42a-3003-4b50-b4e9-0137f683f144","Type":"ContainerStarted","Data":"8bf689f814e2fd94aa53ea73228ff98a7bf9424c85271b3c22164fe795a04373"} Sep 19 09:23:04.855198 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:04.855162 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" event={"ID":"eedad42a-3003-4b50-b4e9-0137f683f144","Type":"ContainerStarted","Data":"282fc1b93436f423d5825c0bd01b4bf79541f2e14d9a0368c95773e9f291e10d"} Sep 19 09:23:04.876887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:04.876842 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" podStartSLOduration=1.796175557 podStartE2EDuration="2.876827754s" totalImagesPullingTime="1.080652197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:23:04.874812755 +0000 UTC m=+991.632933782" watchObservedRunningTime="2026-09-19 09:23:04.876827754 +0000 UTC m=+991.634948778" Sep 19 09:23:12.689134 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:12.689087 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:12.689724 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:12.689166 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:12.701539 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:12.701500 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:12.908896 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:12.908868 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75" Sep 19 09:23:19.776613 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.776570 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml"] Sep 19 09:23:19.781728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.781706 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:19.784406 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.784376 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Sep 19 09:23:19.790111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.790088 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml"] Sep 19 09:23:19.901384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.901347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24058ab2-bda3-4d89-990e-549c45645c1f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:19.901597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.901399 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:19.901597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.901425 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whjml\" (UniqueName: \"kubernetes.io/projected/24058ab2-bda3-4d89-990e-549c45645c1f-kube-api-access-whjml\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:19.901597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.901558 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:19.901706 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.901608 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:19.901706 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:19.901666 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.002803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.002767 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.002983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.002810 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24058ab2-bda3-4d89-990e-549c45645c1f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.002983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.002844 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.002983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.002869 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whjml\" (UniqueName: \"kubernetes.io/projected/24058ab2-bda3-4d89-990e-549c45645c1f-kube-api-access-whjml\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.002983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.002900 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.002983 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.002921 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.003296 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.003271 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.003401 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.003377 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.003573 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.003548 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.005152 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.005132 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24058ab2-bda3-4d89-990e-549c45645c1f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.005902 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.005881 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24058ab2-bda3-4d89-990e-549c45645c1f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.011383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.011339 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whjml\" (UniqueName: \"kubernetes.io/projected/24058ab2-bda3-4d89-990e-549c45645c1f-kube-api-access-whjml\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml\" (UID: \"24058ab2-bda3-4d89-990e-549c45645c1f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.094016 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.093986 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:20.366721 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.366695 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml"] Sep 19 09:23:20.368350 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:23:20.368318 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24058ab2_bda3_4d89_990e_549c45645c1f.slice/crio-cf89786c9e0b05e1b0339a6dbf41cd673a712bd5794d2848084d592e147d932f WatchSource:0}: Error finding container cf89786c9e0b05e1b0339a6dbf41cd673a712bd5794d2848084d592e147d932f: Status 404 returned error can't find the container with id cf89786c9e0b05e1b0339a6dbf41cd673a712bd5794d2848084d592e147d932f Sep 19 09:23:20.940795 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.940734 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" event={"ID":"24058ab2-bda3-4d89-990e-549c45645c1f","Type":"ContainerStarted","Data":"2c3d2efc7cccc8e1fd1edaa6efffb95732dbd95432deb4042a9cb4bbd9443f8f"} Sep 19 09:23:20.941365 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.940810 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" event={"ID":"24058ab2-bda3-4d89-990e-549c45645c1f","Type":"ContainerStarted","Data":"cf89786c9e0b05e1b0339a6dbf41cd673a712bd5794d2848084d592e147d932f"} Sep 19 09:23:20.964283 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:20.964213 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" podStartSLOduration=1.789721718 podStartE2EDuration="1.964191529s" totalImagesPullingTime="174.469811ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:23:20.960967809 +0000 UTC m=+1007.719088818" watchObservedRunningTime="2026-09-19 09:23:20.964191529 +0000 UTC m=+1007.722312556" Sep 19 09:23:30.095073 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:30.095020 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:30.095073 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:30.095080 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:30.107745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:30.107720 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:23:31.000532 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:23:31.000492 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml" Sep 19 09:25:29.878180 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:29.878146 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6479ff984f-b9qsf"] Sep 19 09:25:29.881831 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:29.881812 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:29.894730 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:29.894698 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6479ff984f-b9qsf"] Sep 19 09:25:30.022204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.022173 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf2vp\" (UniqueName: \"kubernetes.io/projected/51bf6be7-3c28-42ea-8153-2d53c67ef456-kube-api-access-mf2vp\") pod \"authorino-6479ff984f-b9qsf\" (UID: \"51bf6be7-3c28-42ea-8153-2d53c67ef456\") " pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:30.022355 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.022240 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51bf6be7-3c28-42ea-8153-2d53c67ef456-tls-cert\") pod \"authorino-6479ff984f-b9qsf\" (UID: \"51bf6be7-3c28-42ea-8153-2d53c67ef456\") " pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:30.123619 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.123574 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51bf6be7-3c28-42ea-8153-2d53c67ef456-tls-cert\") pod \"authorino-6479ff984f-b9qsf\" (UID: \"51bf6be7-3c28-42ea-8153-2d53c67ef456\") " pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:30.123776 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.123671 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf2vp\" (UniqueName: \"kubernetes.io/projected/51bf6be7-3c28-42ea-8153-2d53c67ef456-kube-api-access-mf2vp\") pod \"authorino-6479ff984f-b9qsf\" (UID: \"51bf6be7-3c28-42ea-8153-2d53c67ef456\") " pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:30.126073 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.126048 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/51bf6be7-3c28-42ea-8153-2d53c67ef456-tls-cert\") pod \"authorino-6479ff984f-b9qsf\" (UID: \"51bf6be7-3c28-42ea-8153-2d53c67ef456\") " pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:30.132200 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.132139 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf2vp\" (UniqueName: \"kubernetes.io/projected/51bf6be7-3c28-42ea-8153-2d53c67ef456-kube-api-access-mf2vp\") pod \"authorino-6479ff984f-b9qsf\" (UID: \"51bf6be7-3c28-42ea-8153-2d53c67ef456\") " pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:30.191004 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.190982 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6479ff984f-b9qsf" Sep 19 09:25:30.461696 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.461672 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6479ff984f-b9qsf"] Sep 19 09:25:30.463920 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:25:30.463896 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51bf6be7_3c28_42ea_8153_2d53c67ef456.slice/crio-c826b28f5aab9656b2f657c6ff6e0bf9de51bce5faa4a9ebad4354e3010e579e WatchSource:0}: Error finding container c826b28f5aab9656b2f657c6ff6e0bf9de51bce5faa4a9ebad4354e3010e579e: Status 404 returned error can't find the container with id c826b28f5aab9656b2f657c6ff6e0bf9de51bce5faa4a9ebad4354e3010e579e Sep 19 09:25:30.614466 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:30.614426 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6479ff984f-b9qsf" event={"ID":"51bf6be7-3c28-42ea-8153-2d53c67ef456","Type":"ContainerStarted","Data":"c826b28f5aab9656b2f657c6ff6e0bf9de51bce5faa4a9ebad4354e3010e579e"} Sep 19 09:25:31.623591 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:31.623554 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6479ff984f-b9qsf" event={"ID":"51bf6be7-3c28-42ea-8153-2d53c67ef456","Type":"ContainerStarted","Data":"b23037a4507f928204809f31c53b4765eb6d810596ddde97bab628801f0e73ec"} Sep 19 09:25:31.642925 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:31.642858 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6479ff984f-b9qsf" podStartSLOduration=1.838057117 podStartE2EDuration="2.642836324s" totalImagesPullingTime="804.779207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:25:31.642108687 +0000 UTC m=+1138.400229713" watchObservedRunningTime="2026-09-19 09:25:31.642836324 +0000 UTC m=+1138.400957349" Sep 19 09:25:31.674888 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:31.674852 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55966b5b48-ls2hw"] Sep 19 09:25:31.675102 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:31.675073 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-55966b5b48-ls2hw" podUID="1625a8bd-9b29-44a9-8b9d-f9330f00f962" containerName="authorino" containerID="cri-o://f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0" gracePeriod=30 Sep 19 09:25:31.927109 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:31.927070 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:25:32.041645 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.041611 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1625a8bd-9b29-44a9-8b9d-f9330f00f962-tls-cert\") pod \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " Sep 19 09:25:32.041834 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.041689 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57qnx\" (UniqueName: \"kubernetes.io/projected/1625a8bd-9b29-44a9-8b9d-f9330f00f962-kube-api-access-57qnx\") pod \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\" (UID: \"1625a8bd-9b29-44a9-8b9d-f9330f00f962\") " Sep 19 09:25:32.044234 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.044204 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1625a8bd-9b29-44a9-8b9d-f9330f00f962-kube-api-access-57qnx" (OuterVolumeSpecName: "kube-api-access-57qnx") pod "1625a8bd-9b29-44a9-8b9d-f9330f00f962" (UID: "1625a8bd-9b29-44a9-8b9d-f9330f00f962"). InnerVolumeSpecName "kube-api-access-57qnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:25:32.053314 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.053284 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1625a8bd-9b29-44a9-8b9d-f9330f00f962-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1625a8bd-9b29-44a9-8b9d-f9330f00f962" (UID: "1625a8bd-9b29-44a9-8b9d-f9330f00f962"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:25:32.143370 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.143331 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57qnx\" (UniqueName: \"kubernetes.io/projected/1625a8bd-9b29-44a9-8b9d-f9330f00f962-kube-api-access-57qnx\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:25:32.143370 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.143363 2585 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1625a8bd-9b29-44a9-8b9d-f9330f00f962-tls-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:25:32.630837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.630800 2585 generic.go:356] "Generic (PLEG): container finished" podID="1625a8bd-9b29-44a9-8b9d-f9330f00f962" containerID="f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0" exitCode=0 Sep 19 09:25:32.631216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.630843 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55966b5b48-ls2hw" Sep 19 09:25:32.631216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.630886 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55966b5b48-ls2hw" event={"ID":"1625a8bd-9b29-44a9-8b9d-f9330f00f962","Type":"ContainerDied","Data":"f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0"} Sep 19 09:25:32.631216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.630933 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55966b5b48-ls2hw" event={"ID":"1625a8bd-9b29-44a9-8b9d-f9330f00f962","Type":"ContainerDied","Data":"1352af096cbff10a1373ee04e79af2d73aebfa7675e1d8f59b426a5d98c90060"} Sep 19 09:25:32.631216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.630957 2585 scope.go:117] "RemoveContainer" containerID="f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0" Sep 19 09:25:32.640065 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.640046 2585 scope.go:117] "RemoveContainer" containerID="f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0" Sep 19 09:25:32.640322 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:25:32.640304 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0\": container with ID starting with f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0 not found: ID does not exist" containerID="f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0" Sep 19 09:25:32.640381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.640336 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0"} err="failed to get container status \"f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0\": rpc error: code = NotFound desc = could not find container \"f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0\": container with ID starting with f8834278e64b45c9edf4b0bc77a238c5aecb33f853ce3c5d0bafd96c4cbd59b0 not found: ID does not exist" Sep 19 09:25:32.653584 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.653562 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55966b5b48-ls2hw"] Sep 19 09:25:32.657372 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:32.657352 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55966b5b48-ls2hw"] Sep 19 09:25:33.436062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:33.436029 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1625a8bd-9b29-44a9-8b9d-f9330f00f962" path="/var/lib/kubelet/pods/1625a8bd-9b29-44a9-8b9d-f9330f00f962/volumes" Sep 19 09:25:52.183571 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.183476 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr"] Sep 19 09:25:52.184183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.184162 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1625a8bd-9b29-44a9-8b9d-f9330f00f962" containerName="authorino" Sep 19 09:25:52.184234 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.184189 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1625a8bd-9b29-44a9-8b9d-f9330f00f962" containerName="authorino" Sep 19 09:25:52.184294 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.184283 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1625a8bd-9b29-44a9-8b9d-f9330f00f962" containerName="authorino" Sep 19 09:25:52.187877 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.187854 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.192907 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.192885 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 19 09:25:52.193027 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.192885 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-9dfb0f-openshift-default-dockercfg-zm9sp\"" Sep 19 09:25:52.210785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.210758 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr"] Sep 19 09:25:52.302373 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302341 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-socket\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.302373 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302379 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-envoy\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.302683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302401 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38151ef5-5a51-4648-b24c-4701ad991d4d-istiod-ca-cert\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.302683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302515 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-certs\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.302683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302638 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-data\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.302862 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302696 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-token\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.302862 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302770 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-credential-socket\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.302862 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302828 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-podinfo\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.303279 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.302866 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js949\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-kube-api-access-js949\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404324 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404280 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-data\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404344 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-token\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404390 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-credential-socket\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404437 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-podinfo\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404470 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js949\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-kube-api-access-js949\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404501 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-socket\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404555 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-envoy\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404595 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38151ef5-5a51-4648-b24c-4701ad991d4d-istiod-ca-cert\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404633 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-certs\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.404835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404787 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-data\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.405067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.404914 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-credential-socket\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.405067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.405024 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-certs\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.405382 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.405358 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38151ef5-5a51-4648-b24c-4701ad991d4d-istiod-ca-cert\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.405510 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.405488 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-socket\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.407217 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.407187 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-envoy\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.407683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.407664 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-podinfo\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.419847 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.419814 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js949\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-kube-api-access-js949\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.425649 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.425622 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-token\") pod \"e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.499289 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.499208 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:52.785995 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:52.785971 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr"] Sep 19 09:25:52.787956 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:25:52.787928 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38151ef5_5a51_4648_b24c_4701ad991d4d.slice/crio-a57a6d9d09aedb2ca3451f44f15892eda2b268fbec4652d8c6d0dfb30ecb7397 WatchSource:0}: Error finding container a57a6d9d09aedb2ca3451f44f15892eda2b268fbec4652d8c6d0dfb30ecb7397: Status 404 returned error can't find the container with id a57a6d9d09aedb2ca3451f44f15892eda2b268fbec4652d8c6d0dfb30ecb7397 Sep 19 09:25:53.744234 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:53.744148 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" event={"ID":"38151ef5-5a51-4648-b24c-4701ad991d4d","Type":"ContainerStarted","Data":"a57a6d9d09aedb2ca3451f44f15892eda2b268fbec4652d8c6d0dfb30ecb7397"} Sep 19 09:25:55.655187 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:55.655141 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:25:55.655598 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:55.655243 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:25:55.655598 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:55.655300 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:25:55.761145 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:55.761105 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" event={"ID":"38151ef5-5a51-4648-b24c-4701ad991d4d","Type":"ContainerStarted","Data":"942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359"} Sep 19 09:25:55.793423 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:55.793369 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podStartSLOduration=0.935002754 podStartE2EDuration="3.793354611s" totalImagesPullingTime="2.858351857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:25:55.791233148 +0000 UTC m=+1162.549354172" watchObservedRunningTime="2026-09-19 09:25:55.793354611 +0000 UTC m=+1162.551475636" Sep 19 09:25:56.500421 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.500382 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:56.505707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.505683 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:56.693330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.693297 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx"] Sep 19 09:25:56.696732 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.696715 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.699543 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.699506 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-14e3f6-serving-cert\"" Sep 19 09:25:56.707631 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.707606 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx"] Sep 19 09:25:56.746369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.746335 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dx6v\" (UniqueName: \"kubernetes.io/projected/e429751d-4a55-4aa3-a9f9-0c62c669045c-kube-api-access-2dx6v\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.746584 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.746390 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-metrics-certs\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.746584 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.746481 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.746584 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.746502 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e429751d-4a55-4aa3-a9f9-0c62c669045c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.768435 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.768222 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:56.769221 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.769195 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:25:56.847353 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.847318 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.847602 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.847383 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e429751d-4a55-4aa3-a9f9-0c62c669045c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.847602 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:25:56.847394 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-serving-cert: secret "maas-api-e2e-worker-w0-14e3f6-serving-cert" not found Sep 19 09:25:56.847602 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:25:56.847466 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls podName:e429751d-4a55-4aa3-a9f9-0c62c669045c nodeName:}" failed. No retries permitted until 2026-09-19 09:25:57.347447633 +0000 UTC m=+1164.105568640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls") pod "maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" (UID: "e429751d-4a55-4aa3-a9f9-0c62c669045c") : secret "maas-api-e2e-worker-w0-14e3f6-serving-cert" not found Sep 19 09:25:56.847602 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.847559 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dx6v\" (UniqueName: \"kubernetes.io/projected/e429751d-4a55-4aa3-a9f9-0c62c669045c-kube-api-access-2dx6v\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.847852 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.847753 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-metrics-certs\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.848050 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.848026 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e429751d-4a55-4aa3-a9f9-0c62c669045c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.850156 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.850131 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-metrics-certs\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:56.865416 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:56.865392 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dx6v\" (UniqueName: \"kubernetes.io/projected/e429751d-4a55-4aa3-a9f9-0c62c669045c-kube-api-access-2dx6v\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:57.352402 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:57.352367 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:57.354879 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:57.354856 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:57.607634 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:57.607508 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:25:57.891569 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:57.891541 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx"] Sep 19 09:25:57.892167 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:25:57.892143 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode429751d_4a55_4aa3_a9f9_0c62c669045c.slice/crio-bc87cffa46f22cac422366578f04fad679cc0b8e3d660c3a30186cbb167fae01 WatchSource:0}: Error finding container bc87cffa46f22cac422366578f04fad679cc0b8e3d660c3a30186cbb167fae01: Status 404 returned error can't find the container with id bc87cffa46f22cac422366578f04fad679cc0b8e3d660c3a30186cbb167fae01 Sep 19 09:25:58.057685 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.057647 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67"] Sep 19 09:25:58.062743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.062720 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.066553 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.066533 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-acc640-serving-cert\"" Sep 19 09:25:58.084779 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.084750 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67"] Sep 19 09:25:58.159438 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.159345 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-metrics-certs\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.159607 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.159460 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfqqs\" (UniqueName: \"kubernetes.io/projected/9baf916d-70a1-4cab-b1f6-32024558c917-kube-api-access-qfqqs\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.159607 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.159495 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baf916d-70a1-4cab-b1f6-32024558c917-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.159607 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.159540 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.261130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.261092 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.261311 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.261195 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-metrics-certs\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.261311 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:25:58.261259 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-serving-cert: secret "maas-api-e2e-worker-w4-acc640-serving-cert" not found Sep 19 09:25:58.261428 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:25:58.261337 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls podName:9baf916d-70a1-4cab-b1f6-32024558c917 nodeName:}" failed. No retries permitted until 2026-09-19 09:25:58.761317571 +0000 UTC m=+1165.519438575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls") pod "maas-api-e2e-worker-w4-acc640-8466649555-6xg67" (UID: "9baf916d-70a1-4cab-b1f6-32024558c917") : secret "maas-api-e2e-worker-w4-acc640-serving-cert" not found Sep 19 09:25:58.261428 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.261269 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqqs\" (UniqueName: \"kubernetes.io/projected/9baf916d-70a1-4cab-b1f6-32024558c917-kube-api-access-qfqqs\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.261566 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.261457 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baf916d-70a1-4cab-b1f6-32024558c917-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.262283 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.262257 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baf916d-70a1-4cab-b1f6-32024558c917-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.264134 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.264109 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-metrics-certs\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.273495 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.273472 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfqqs\" (UniqueName: \"kubernetes.io/projected/9baf916d-70a1-4cab-b1f6-32024558c917-kube-api-access-qfqqs\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.767253 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.767219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.769632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.769614 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls\") pod \"maas-api-e2e-worker-w4-acc640-8466649555-6xg67\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:58.780757 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.780726 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" event={"ID":"e429751d-4a55-4aa3-a9f9-0c62c669045c","Type":"ContainerStarted","Data":"2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8"} Sep 19 09:25:58.780890 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.780775 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" event={"ID":"e429751d-4a55-4aa3-a9f9-0c62c669045c","Type":"ContainerStarted","Data":"bc87cffa46f22cac422366578f04fad679cc0b8e3d660c3a30186cbb167fae01"} Sep 19 09:25:58.822272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.822217 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" podStartSLOduration=2.423667715 podStartE2EDuration="2.822200518s" totalImagesPullingTime="398.532803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:25:58.821358914 +0000 UTC m=+1165.579479941" watchObservedRunningTime="2026-09-19 09:25:58.822200518 +0000 UTC m=+1165.580321542" Sep 19 09:25:58.975837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:58.975799 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:25:59.399386 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:59.399304 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67"] Sep 19 09:25:59.402243 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:25:59.402210 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baf916d_70a1_4cab_b1f6_32024558c917.slice/crio-bc316ad5077aea010bd6461680e57514fba063cbf7be8b5625dedfd87535b4d6 WatchSource:0}: Error finding container bc316ad5077aea010bd6461680e57514fba063cbf7be8b5625dedfd87535b4d6: Status 404 returned error can't find the container with id bc316ad5077aea010bd6461680e57514fba063cbf7be8b5625dedfd87535b4d6 Sep 19 09:25:59.787643 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:59.787617 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" event={"ID":"9baf916d-70a1-4cab-b1f6-32024558c917","Type":"ContainerStarted","Data":"bc316ad5077aea010bd6461680e57514fba063cbf7be8b5625dedfd87535b4d6"} Sep 19 09:25:59.787762 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:25:59.787749 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:26:00.795061 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:00.795014 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" event={"ID":"9baf916d-70a1-4cab-b1f6-32024558c917","Type":"ContainerStarted","Data":"f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724"} Sep 19 09:26:00.795571 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:00.795340 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:26:00.824173 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:00.824126 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" podStartSLOduration=2.466955519 podStartE2EDuration="2.824110705s" totalImagesPullingTime="357.155186ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:00.821822292 +0000 UTC m=+1167.579943340" watchObservedRunningTime="2026-09-19 09:26:00.824110705 +0000 UTC m=+1167.582231730" Sep 19 09:26:02.091543 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.091492 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw"] Sep 19 09:26:02.095699 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.095678 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.105720 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.104498 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9dfb0f-serving-cert\"" Sep 19 09:26:02.120434 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.120398 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw"] Sep 19 09:26:02.199148 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.199112 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.199148 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.199147 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6ljz\" (UniqueName: \"kubernetes.io/projected/aee54331-9423-4274-83e5-2320e67245a8-kube-api-access-j6ljz\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.199422 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.199301 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-metrics-certs\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.199422 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.199399 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee54331-9423-4274-83e5-2320e67245a8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.300673 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.300626 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-metrics-certs\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.300887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.300720 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee54331-9423-4274-83e5-2320e67245a8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.300887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.300749 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.300887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.300770 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6ljz\" (UniqueName: \"kubernetes.io/projected/aee54331-9423-4274-83e5-2320e67245a8-kube-api-access-j6ljz\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.301069 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:26:02.300886 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-serving-cert: secret "maas-api-e2e-inf-a-9dfb0f-serving-cert" not found Sep 19 09:26:02.301069 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:26:02.300959 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls podName:aee54331-9423-4274-83e5-2320e67245a8 nodeName:}" failed. No retries permitted until 2026-09-19 09:26:02.800942509 +0000 UTC m=+1169.559063512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls") pod "maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" (UID: "aee54331-9423-4274-83e5-2320e67245a8") : secret "maas-api-e2e-inf-a-9dfb0f-serving-cert" not found Sep 19 09:26:02.301461 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.301440 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee54331-9423-4274-83e5-2320e67245a8-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.303186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.303153 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-metrics-certs\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.310138 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.310107 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6ljz\" (UniqueName: \"kubernetes.io/projected/aee54331-9423-4274-83e5-2320e67245a8-kube-api-access-j6ljz\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.806066 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.806029 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:02.808461 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:02.808438 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls\") pod \"maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:03.012261 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.012223 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:03.092183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.089950 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58"] Sep 19 09:26:03.107282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.106424 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.124692 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.124462 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-9dfb0f-dockercfg-8tchb\"" Sep 19 09:26:03.126776 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.126595 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-9dfb0f\"" Sep 19 09:26:03.133655 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.133441 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58"] Sep 19 09:26:03.213051 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.213015 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.213249 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.213110 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9s2f\" (UniqueName: \"kubernetes.io/projected/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-kube-api-access-s9s2f\") pod \"payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.315075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.314341 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.315075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.314434 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9s2f\" (UniqueName: \"kubernetes.io/projected/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-kube-api-access-s9s2f\") pod \"payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.315384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.315268 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.326202 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.326167 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9s2f\" (UniqueName: \"kubernetes.io/projected/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-kube-api-access-s9s2f\") pod \"payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.346208 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.346175 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw"] Sep 19 09:26:03.347578 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:26:03.347553 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee54331_9423_4274_83e5_2320e67245a8.slice/crio-633eefa11dd187f39065eafd69d1605a36b97011b18f45dc66279b7db3572f8d WatchSource:0}: Error finding container 633eefa11dd187f39065eafd69d1605a36b97011b18f45dc66279b7db3572f8d: Status 404 returned error can't find the container with id 633eefa11dd187f39065eafd69d1605a36b97011b18f45dc66279b7db3572f8d Sep 19 09:26:03.424767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.424722 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:03.720562 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.720530 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58"] Sep 19 09:26:03.723830 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:26:03.723795 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode11cc70b_3ed3_412d_bd16_9e646d3c4a52.slice/crio-805736656ba5d1f1a9b0a52f0d09a504158155c696107a32851cb28afd1c21f8 WatchSource:0}: Error finding container 805736656ba5d1f1a9b0a52f0d09a504158155c696107a32851cb28afd1c21f8: Status 404 returned error can't find the container with id 805736656ba5d1f1a9b0a52f0d09a504158155c696107a32851cb28afd1c21f8 Sep 19 09:26:03.815043 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.814999 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" event={"ID":"e11cc70b-3ed3-412d-bd16-9e646d3c4a52","Type":"ContainerStarted","Data":"805736656ba5d1f1a9b0a52f0d09a504158155c696107a32851cb28afd1c21f8"} Sep 19 09:26:03.817667 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.817397 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" event={"ID":"aee54331-9423-4274-83e5-2320e67245a8","Type":"ContainerStarted","Data":"e9a7f9ab284c1934427901c3dbbf935036c8a9ac1b138fee3b1c45485b859a83"} Sep 19 09:26:03.817667 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.817628 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:03.817853 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.817674 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" event={"ID":"aee54331-9423-4274-83e5-2320e67245a8","Type":"ContainerStarted","Data":"633eefa11dd187f39065eafd69d1605a36b97011b18f45dc66279b7db3572f8d"} Sep 19 09:26:03.844644 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:03.844586 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" podStartSLOduration=1.513066685 podStartE2EDuration="1.844570041s" totalImagesPullingTime="331.503356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:03.842046236 +0000 UTC m=+1170.600167259" watchObservedRunningTime="2026-09-19 09:26:03.844570041 +0000 UTC m=+1170.602691086" Sep 19 09:26:05.802868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:05.802838 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:26:05.831540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:05.831476 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" event={"ID":"e11cc70b-3ed3-412d-bd16-9e646d3c4a52","Type":"ContainerStarted","Data":"c2bc8c2574862e7c229d688be1a587257b7abb02245bde3e5eb9e16eed011db1"} Sep 19 09:26:05.831728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:05.831563 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:05.850862 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:05.850809 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" podStartSLOduration=1.768813226 podStartE2EDuration="2.850791674s" totalImagesPullingTime="1.081978448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:05.85057072 +0000 UTC m=+1172.608691745" watchObservedRunningTime="2026-09-19 09:26:05.850791674 +0000 UTC m=+1172.608912755" Sep 19 09:26:06.806563 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:06.806513 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:26:07.811652 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.811613 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj"] Sep 19 09:26:07.817036 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.817012 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:07.819447 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.819421 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-0ce808-serving-cert\"" Sep 19 09:26:07.824960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.824929 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj"] Sep 19 09:26:07.962355 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.962319 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkgpg\" (UniqueName: \"kubernetes.io/projected/c2d8f377-82f8-45c2-9aad-5df49d442e7c-kube-api-access-gkgpg\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:07.962355 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.962357 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d8f377-82f8-45c2-9aad-5df49d442e7c-openshift-service-ca\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:07.962610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.962394 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-metrics-certs\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:07.962610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:07.962468 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-maas-api-tls\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.063717 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.063643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-metrics-certs\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.063717 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.063687 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-maas-api-tls\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.063929 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.063755 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkgpg\" (UniqueName: \"kubernetes.io/projected/c2d8f377-82f8-45c2-9aad-5df49d442e7c-kube-api-access-gkgpg\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.063929 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.063856 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d8f377-82f8-45c2-9aad-5df49d442e7c-openshift-service-ca\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.064491 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.064471 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d8f377-82f8-45c2-9aad-5df49d442e7c-openshift-service-ca\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.066321 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.066304 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-metrics-certs\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.066381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.066366 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-maas-api-tls\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.078700 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.078679 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkgpg\" (UniqueName: \"kubernetes.io/projected/c2d8f377-82f8-45c2-9aad-5df49d442e7c-kube-api-access-gkgpg\") pod \"maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.130934 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.130900 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.405553 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.405508 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj"] Sep 19 09:26:08.407259 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:26:08.407231 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d8f377_82f8_45c2_9aad_5df49d442e7c.slice/crio-2dd00d0a0e47f54a6c8b4060dbd43cb7a9248769a37eba3d1de1e4eb66797169 WatchSource:0}: Error finding container 2dd00d0a0e47f54a6c8b4060dbd43cb7a9248769a37eba3d1de1e4eb66797169: Status 404 returned error can't find the container with id 2dd00d0a0e47f54a6c8b4060dbd43cb7a9248769a37eba3d1de1e4eb66797169 Sep 19 09:26:08.852229 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.852192 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" event={"ID":"c2d8f377-82f8-45c2-9aad-5df49d442e7c","Type":"ContainerStarted","Data":"69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a"} Sep 19 09:26:08.852694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.852250 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" event={"ID":"c2d8f377-82f8-45c2-9aad-5df49d442e7c","Type":"ContainerStarted","Data":"2dd00d0a0e47f54a6c8b4060dbd43cb7a9248769a37eba3d1de1e4eb66797169"} Sep 19 09:26:08.852694 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.852276 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:08.870887 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:08.870830 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" podStartSLOduration=1.526705435 podStartE2EDuration="1.8708092s" totalImagesPullingTime="344.103765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:08.869473511 +0000 UTC m=+1175.627594537" watchObservedRunningTime="2026-09-19 09:26:08.8708092 +0000 UTC m=+1175.628930224" Sep 19 09:26:09.828960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:09.828926 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:26:10.034133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.034096 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p"] Sep 19 09:26:10.038878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.038856 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.043745 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.043721 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b272f719-serving-cert\"" Sep 19 09:26:10.075209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.075175 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p"] Sep 19 09:26:10.184958 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.184864 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.185120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.185021 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1263921-3dd0-4fe8-a680-17b07a74237b-openshift-service-ca\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.185120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.185070 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-metrics-certs\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.185120 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.185101 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sls8z\" (UniqueName: \"kubernetes.io/projected/c1263921-3dd0-4fe8-a680-17b07a74237b-kube-api-access-sls8z\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.286505 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.286471 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.286723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.286594 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1263921-3dd0-4fe8-a680-17b07a74237b-openshift-service-ca\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.286723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.286619 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-metrics-certs\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.286723 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:26:10.286623 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-serving-cert: secret "maas-api-e2e-ait-b272f719-serving-cert" not found Sep 19 09:26:10.286723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.286642 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sls8z\" (UniqueName: \"kubernetes.io/projected/c1263921-3dd0-4fe8-a680-17b07a74237b-kube-api-access-sls8z\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.286723 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:26:10.286714 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls podName:c1263921-3dd0-4fe8-a680-17b07a74237b nodeName:}" failed. No retries permitted until 2026-09-19 09:26:10.786690102 +0000 UTC m=+1177.544811121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls") pod "maas-api-e2e-ait-b272f719-757785b864-pm88p" (UID: "c1263921-3dd0-4fe8-a680-17b07a74237b") : secret "maas-api-e2e-ait-b272f719-serving-cert" not found Sep 19 09:26:10.287239 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.287214 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1263921-3dd0-4fe8-a680-17b07a74237b-openshift-service-ca\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.289148 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.289120 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-metrics-certs\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.295625 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.295591 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sls8z\" (UniqueName: \"kubernetes.io/projected/c1263921-3dd0-4fe8-a680-17b07a74237b-kube-api-access-sls8z\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.791286 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.791242 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.793608 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.793589 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls\") pod \"maas-api-e2e-ait-b272f719-757785b864-pm88p\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:10.951707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:10.951674 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:11.229905 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:11.229881 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p"] Sep 19 09:26:11.232401 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:26:11.232376 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1263921_3dd0_4fe8_a680_17b07a74237b.slice/crio-2ebb0e7e198cef958bc5fdf56fba1fc984c600543fc1de95e6a7fa98eb08170f WatchSource:0}: Error finding container 2ebb0e7e198cef958bc5fdf56fba1fc984c600543fc1de95e6a7fa98eb08170f: Status 404 returned error can't find the container with id 2ebb0e7e198cef958bc5fdf56fba1fc984c600543fc1de95e6a7fa98eb08170f Sep 19 09:26:11.871398 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:11.871363 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" event={"ID":"c1263921-3dd0-4fe8-a680-17b07a74237b","Type":"ContainerStarted","Data":"9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7"} Sep 19 09:26:11.871614 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:11.871426 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" event={"ID":"c1263921-3dd0-4fe8-a680-17b07a74237b","Type":"ContainerStarted","Data":"2ebb0e7e198cef958bc5fdf56fba1fc984c600543fc1de95e6a7fa98eb08170f"} Sep 19 09:26:11.871614 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:11.871545 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:11.890643 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:11.890589 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" podStartSLOduration=2.571873532 podStartE2EDuration="2.890569905s" totalImagesPullingTime="318.696373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:11.889064503 +0000 UTC m=+1178.647185542" watchObservedRunningTime="2026-09-19 09:26:11.890569905 +0000 UTC m=+1178.648690934" Sep 19 09:26:14.864936 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:14.864900 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:26:16.840724 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:16.840692 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:26:17.883398 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:17.883369 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:18.158332 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.158257 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx"] Sep 19 09:26:18.163111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.163089 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.165457 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.165436 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c18110-openshift-default-dockercfg-kfslx\"" Sep 19 09:26:18.179954 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.179928 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx"] Sep 19 09:26:18.281322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281270 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-data\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281346 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-socket\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281510 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281425 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-token\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281574 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281508 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-podinfo\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281596 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-certs\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281720 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281705 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-envoy\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281753 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpmh7\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-kube-api-access-tpmh7\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281823 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281809 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-credential-socket\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.281878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.281865 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istiod-ca-cert\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.382868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.382837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-token\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.382880 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-podinfo\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.382901 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-certs\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.382923 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-envoy\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.382954 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpmh7\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-kube-api-access-tpmh7\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.382976 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-credential-socket\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.382991 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istiod-ca-cert\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.383027 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-data\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.383054 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-socket\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.383359 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.383344 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-socket\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.384068 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.383810 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-credential-socket\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.384068 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.383885 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-data\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.384250 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.384108 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-certs\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.384724 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.384700 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istiod-ca-cert\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.386304 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.386203 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-envoy\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.386407 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.386380 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-podinfo\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.392795 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.392775 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpmh7\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-kube-api-access-tpmh7\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.393064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.393043 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-token\") pod \"e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.474608 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.474507 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:18.790252 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.790225 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx"] Sep 19 09:26:18.792592 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:26:18.792538 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36ec3b2_5604_468e_8cec_f5b4160cfa09.slice/crio-9975d0ecf90a1c598d123ccdb13415471795528572891391fcf46a0d54798be3 WatchSource:0}: Error finding container 9975d0ecf90a1c598d123ccdb13415471795528572891391fcf46a0d54798be3: Status 404 returned error can't find the container with id 9975d0ecf90a1c598d123ccdb13415471795528572891391fcf46a0d54798be3 Sep 19 09:26:18.795414 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.795375 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:18.795511 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.795475 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:18.795580 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.795537 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:18.916723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.915716 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" event={"ID":"d36ec3b2-5604-468e-8cec-f5b4160cfa09","Type":"ContainerStarted","Data":"5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555"} Sep 19 09:26:18.916723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.915780 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" event={"ID":"d36ec3b2-5604-468e-8cec-f5b4160cfa09","Type":"ContainerStarted","Data":"9975d0ecf90a1c598d123ccdb13415471795528572891391fcf46a0d54798be3"} Sep 19 09:26:18.941763 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:18.941708 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podStartSLOduration=0.941692531 podStartE2EDuration="941.692531ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:18.937637953 +0000 UTC m=+1185.695758977" watchObservedRunningTime="2026-09-19 09:26:18.941692531 +0000 UTC m=+1185.699813555" Sep 19 09:26:19.475295 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:19.475216 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:19.480270 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:19.480244 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:19.923850 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:19.923811 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:19.924943 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:19.924910 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:22.192692 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.192651 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv"] Sep 19 09:26:22.195808 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.195786 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.198571 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.198547 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c18110-serving-cert\"" Sep 19 09:26:22.213047 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.213026 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv"] Sep 19 09:26:22.219814 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.219697 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj8sb\" (UniqueName: \"kubernetes.io/projected/9b2591f3-cdf3-49e3-a269-c8644d908a60-kube-api-access-bj8sb\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.219814 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.219747 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-metrics-certs\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.220000 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.219870 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2591f3-cdf3-49e3-a269-c8644d908a60-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.220000 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.219933 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.321256 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.321225 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2591f3-cdf3-49e3-a269-c8644d908a60-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.321417 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.321294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.321417 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.321367 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj8sb\" (UniqueName: \"kubernetes.io/projected/9b2591f3-cdf3-49e3-a269-c8644d908a60-kube-api-access-bj8sb\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.321417 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.321393 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-metrics-certs\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.321629 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:26:22.321470 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-serving-cert: secret "maas-api-e2e-inf-b-c18110-serving-cert" not found Sep 19 09:26:22.321629 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:26:22.321580 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls podName:9b2591f3-cdf3-49e3-a269-c8644d908a60 nodeName:}" failed. No retries permitted until 2026-09-19 09:26:22.821558706 +0000 UTC m=+1189.579679722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls") pod "maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" (UID: "9b2591f3-cdf3-49e3-a269-c8644d908a60") : secret "maas-api-e2e-inf-b-c18110-serving-cert" not found Sep 19 09:26:22.321888 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.321861 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2591f3-cdf3-49e3-a269-c8644d908a60-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.324434 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.324413 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-metrics-certs\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.330202 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.330182 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj8sb\" (UniqueName: \"kubernetes.io/projected/9b2591f3-cdf3-49e3-a269-c8644d908a60-kube-api-access-bj8sb\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.826946 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.826913 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.829330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.829306 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls\") pod \"maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:22.943047 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.943013 2585 generic.go:356] "Generic (PLEG): container finished" podID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerID="942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359" exitCode=0 Sep 19 09:26:22.943208 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.943082 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" event={"ID":"38151ef5-5a51-4648-b24c-4701ad991d4d","Type":"ContainerDied","Data":"942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359"} Sep 19 09:26:22.943475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.943460 2585 scope.go:117] "RemoveContainer" containerID="942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359" Sep 19 09:26:22.944773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.944743 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:22.944859 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.944816 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:22.944859 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:22.944846 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:23.107619 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:23.107507 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:23.391147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:23.391111 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv"] Sep 19 09:26:23.391955 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:26:23.391919 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b2591f3_cdf3_49e3_a269_c8644d908a60.slice/crio-6a8c59bf174c984e189f9c35f526d3aed762bf2cc6125f1d6f5401e0b8fb8f34 WatchSource:0}: Error finding container 6a8c59bf174c984e189f9c35f526d3aed762bf2cc6125f1d6f5401e0b8fb8f34: Status 404 returned error can't find the container with id 6a8c59bf174c984e189f9c35f526d3aed762bf2cc6125f1d6f5401e0b8fb8f34 Sep 19 09:26:23.953992 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:23.953904 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" event={"ID":"38151ef5-5a51-4648-b24c-4701ad991d4d","Type":"ContainerStarted","Data":"2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4"} Sep 19 09:26:23.955785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:23.955762 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" event={"ID":"9b2591f3-cdf3-49e3-a269-c8644d908a60","Type":"ContainerStarted","Data":"866bce3357cbca4c89251ebe98c1f3721f66b6676ab77a337831284bc6b6889f"} Sep 19 09:26:23.955914 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:23.955810 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" event={"ID":"9b2591f3-cdf3-49e3-a269-c8644d908a60","Type":"ContainerStarted","Data":"6a8c59bf174c984e189f9c35f526d3aed762bf2cc6125f1d6f5401e0b8fb8f34"} Sep 19 09:26:23.955914 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:23.955838 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:23.993262 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:23.993214 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" podStartSLOduration=1.7219451970000001 podStartE2EDuration="1.993194576s" totalImagesPullingTime="271.249379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:23.990009908 +0000 UTC m=+1190.748130933" watchObservedRunningTime="2026-09-19 09:26:23.993194576 +0000 UTC m=+1190.751315603" Sep 19 09:26:24.499590 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:24.499553 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:26:24.500459 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:24.500431 2585 patch_prober.go:28] interesting pod/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 19 09:26:24.500601 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:24.500488 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 19 09:26:25.500327 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:25.500289 2585 patch_prober.go:28] interesting pod/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 19 09:26:25.500814 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:25.500358 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 19 09:26:26.500581 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:26.500545 2585 patch_prober.go:28] interesting pod/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 19 09:26:26.500950 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:26.500613 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 19 09:26:26.768666 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:26.768570 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:26:27.499647 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:27.499604 2585 patch_prober.go:28] interesting pod/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 19 09:26:27.499860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:27.499663 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 19 09:26:28.500340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:28.500308 2585 patch_prober.go:28] interesting pod/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 19 09:26:28.500717 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:28.500370 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 19 09:26:29.499624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:29.499592 2585 patch_prober.go:28] interesting pod/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Sep 19 09:26:29.499801 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:29.499642 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Sep 19 09:26:29.968410 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:29.968381 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:26:30.503719 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:30.503687 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:26:30.504713 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:30.504692 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:26:33.972723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:33.972693 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p"] Sep 19 09:26:33.973166 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:33.972913 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" podUID="c1263921-3dd0-4fe8-a680-17b07a74237b" containerName="maas-api" containerID="cri-o://9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7" gracePeriod=30 Sep 19 09:26:34.236719 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.236657 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:34.340608 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.340573 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1263921-3dd0-4fe8-a680-17b07a74237b-openshift-service-ca\") pod \"c1263921-3dd0-4fe8-a680-17b07a74237b\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " Sep 19 09:26:34.340608 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.340619 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sls8z\" (UniqueName: \"kubernetes.io/projected/c1263921-3dd0-4fe8-a680-17b07a74237b-kube-api-access-sls8z\") pod \"c1263921-3dd0-4fe8-a680-17b07a74237b\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " Sep 19 09:26:34.340870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.340725 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls\") pod \"c1263921-3dd0-4fe8-a680-17b07a74237b\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " Sep 19 09:26:34.340870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.340819 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-metrics-certs\") pod \"c1263921-3dd0-4fe8-a680-17b07a74237b\" (UID: \"c1263921-3dd0-4fe8-a680-17b07a74237b\") " Sep 19 09:26:34.341933 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.341887 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1263921-3dd0-4fe8-a680-17b07a74237b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1263921-3dd0-4fe8-a680-17b07a74237b" (UID: "c1263921-3dd0-4fe8-a680-17b07a74237b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:26:34.344827 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.344673 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c1263921-3dd0-4fe8-a680-17b07a74237b" (UID: "c1263921-3dd0-4fe8-a680-17b07a74237b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:26:34.345082 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.345052 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1263921-3dd0-4fe8-a680-17b07a74237b" (UID: "c1263921-3dd0-4fe8-a680-17b07a74237b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:26:34.345082 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.345060 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1263921-3dd0-4fe8-a680-17b07a74237b-kube-api-access-sls8z" (OuterVolumeSpecName: "kube-api-access-sls8z") pod "c1263921-3dd0-4fe8-a680-17b07a74237b" (UID: "c1263921-3dd0-4fe8-a680-17b07a74237b"). InnerVolumeSpecName "kube-api-access-sls8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:26:34.441782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.441747 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:26:34.441782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.441778 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c1263921-3dd0-4fe8-a680-17b07a74237b-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:26:34.442068 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.441792 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1263921-3dd0-4fe8-a680-17b07a74237b-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:26:34.442068 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:34.441807 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sls8z\" (UniqueName: \"kubernetes.io/projected/c1263921-3dd0-4fe8-a680-17b07a74237b-kube-api-access-sls8z\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:26:35.030419 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.030383 2585 generic.go:356] "Generic (PLEG): container finished" podID="c1263921-3dd0-4fe8-a680-17b07a74237b" containerID="9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7" exitCode=0 Sep 19 09:26:35.030911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.030428 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" Sep 19 09:26:35.030911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.030452 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" event={"ID":"c1263921-3dd0-4fe8-a680-17b07a74237b","Type":"ContainerDied","Data":"9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7"} Sep 19 09:26:35.030911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.030505 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p" event={"ID":"c1263921-3dd0-4fe8-a680-17b07a74237b","Type":"ContainerDied","Data":"2ebb0e7e198cef958bc5fdf56fba1fc984c600543fc1de95e6a7fa98eb08170f"} Sep 19 09:26:35.030911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.030568 2585 scope.go:117] "RemoveContainer" containerID="9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7" Sep 19 09:26:35.040716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.040691 2585 scope.go:117] "RemoveContainer" containerID="9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7" Sep 19 09:26:35.041053 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:26:35.041017 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7\": container with ID starting with 9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7 not found: ID does not exist" containerID="9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7" Sep 19 09:26:35.041133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.041061 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7"} err="failed to get container status \"9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7\": rpc error: code = NotFound desc = could not find container \"9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7\": container with ID starting with 9bdf8a415db42f80ac78a3948cce53fc726d41ea8aff90faf11f6f89600945d7 not found: ID does not exist" Sep 19 09:26:35.069738 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.069707 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p"] Sep 19 09:26:35.077698 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.077679 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b272f719-757785b864-pm88p"] Sep 19 09:26:35.436735 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.436703 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1263921-3dd0-4fe8-a680-17b07a74237b" path="/var/lib/kubelet/pods/c1263921-3dd0-4fe8-a680-17b07a74237b/volumes" Sep 19 09:26:35.482494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.482463 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:26:35.485315 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.485290 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:26:35.487350 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.487331 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:26:35.489930 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:35.489909 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:26:47.105708 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:47.105674 2585 generic.go:356] "Generic (PLEG): container finished" podID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerID="5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555" exitCode=0 Sep 19 09:26:47.106114 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:47.105749 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" event={"ID":"d36ec3b2-5604-468e-8cec-f5b4160cfa09","Type":"ContainerDied","Data":"5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555"} Sep 19 09:26:47.106159 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:47.106114 2585 scope.go:117] "RemoveContainer" containerID="5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555" Sep 19 09:26:47.107288 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:47.107263 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:47.107345 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:47.107333 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:47.107396 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:47.107384 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:26:48.114618 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:48.114571 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" event={"ID":"d36ec3b2-5604-468e-8cec-f5b4160cfa09","Type":"ContainerStarted","Data":"267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d"} Sep 19 09:26:48.474768 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:48.474676 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:48.476186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:48.476152 2585 patch_prober.go:28] interesting pod/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Sep 19 09:26:48.476324 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:48.476219 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Sep 19 09:26:49.475871 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:49.475832 2585 patch_prober.go:28] interesting pod/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Sep 19 09:26:49.476280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:49.475890 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Sep 19 09:26:49.924047 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:49.924007 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:50.474878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:50.474839 2585 patch_prober.go:28] interesting pod/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Sep 19 09:26:50.475069 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:50.474907 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Sep 19 09:26:51.474800 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:51.474769 2585 patch_prober.go:28] interesting pod/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Sep 19 09:26:51.475145 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:51.474818 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Sep 19 09:26:52.475825 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:52.475781 2585 patch_prober.go:28] interesting pod/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Sep 19 09:26:52.476334 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:52.475836 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Sep 19 09:26:53.478938 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:53.478911 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:26:53.479961 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:26:53.479945 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:27:58.359610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.359575 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp"] Sep 19 09:27:58.360100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.360041 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1263921-3dd0-4fe8-a680-17b07a74237b" containerName="maas-api" Sep 19 09:27:58.360100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.360054 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1263921-3dd0-4fe8-a680-17b07a74237b" containerName="maas-api" Sep 19 09:27:58.360185 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.360153 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1263921-3dd0-4fe8-a680-17b07a74237b" containerName="maas-api" Sep 19 09:27:58.363280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.363263 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.365767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.365744 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-61e32580-openshift-default-dockercfg-nnkkx\"" Sep 19 09:27:58.379297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.379271 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp"] Sep 19 09:27:58.455271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455232 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-socket\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455268 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-podinfo\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455296 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-token\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455411 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-envoy\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455441 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2txk\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-kube-api-access-m2txk\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455461 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23da6235-dc15-4820-8f9a-ba0bdf148689-istiod-ca-cert\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455547 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-certs\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455824 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455580 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-data\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.455824 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.455607 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-credential-socket\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556352 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556316 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-certs\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556352 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556354 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-data\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556380 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-credential-socket\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556430 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-socket\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556454 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-podinfo\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556482 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-token\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556822 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556756 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-certs\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556822 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556787 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-data\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556933 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556851 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-credential-socket\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.556933 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556877 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-socket\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.557039 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556928 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-envoy\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.557039 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556972 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2txk\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-kube-api-access-m2txk\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.557039 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.556993 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23da6235-dc15-4820-8f9a-ba0bdf148689-istiod-ca-cert\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.557598 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.557575 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23da6235-dc15-4820-8f9a-ba0bdf148689-istiod-ca-cert\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.558998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.558965 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-envoy\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.559288 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.559270 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-podinfo\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.565172 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.565144 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2txk\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-kube-api-access-m2txk\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.565632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.565613 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-token\") pod \"e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.675562 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.675453 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:27:58.961867 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.961834 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp"] Sep 19 09:27:58.963440 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:27:58.963415 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23da6235_dc15_4820_8f9a_ba0bdf148689.slice/crio-873bc467e0fc6952aa1f355f82617a994d1fc28520460ed6fbb04f2d59dbefd8 WatchSource:0}: Error finding container 873bc467e0fc6952aa1f355f82617a994d1fc28520460ed6fbb04f2d59dbefd8: Status 404 returned error can't find the container with id 873bc467e0fc6952aa1f355f82617a994d1fc28520460ed6fbb04f2d59dbefd8 Sep 19 09:27:58.966199 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.966167 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:27:58.966302 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.966239 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:27:58.966347 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:58.966302 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:27:59.537834 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:59.537797 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" event={"ID":"23da6235-dc15-4820-8f9a-ba0bdf148689","Type":"ContainerStarted","Data":"e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5"} Sep 19 09:27:59.538332 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:59.537853 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" event={"ID":"23da6235-dc15-4820-8f9a-ba0bdf148689","Type":"ContainerStarted","Data":"873bc467e0fc6952aa1f355f82617a994d1fc28520460ed6fbb04f2d59dbefd8"} Sep 19 09:27:59.560585 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:59.560539 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" podStartSLOduration=1.5605052590000001 podStartE2EDuration="1.560505259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:59.558143342 +0000 UTC m=+1286.316264365" watchObservedRunningTime="2026-09-19 09:27:59.560505259 +0000 UTC m=+1286.318626283" Sep 19 09:27:59.676655 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:27:59.676610 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:00.681139 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:00.681103 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:01.550740 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:01.550706 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:01.551807 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:01.551785 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:17.277109 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.277076 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj"] Sep 19 09:28:17.277669 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.277361 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" podUID="c2d8f377-82f8-45c2-9aad-5df49d442e7c" containerName="maas-api" containerID="cri-o://69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a" gracePeriod=30 Sep 19 09:28:17.620541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.617686 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:28:17.659962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.659499 2585 generic.go:356] "Generic (PLEG): container finished" podID="c2d8f377-82f8-45c2-9aad-5df49d442e7c" containerID="69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a" exitCode=0 Sep 19 09:28:17.659962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.659568 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" event={"ID":"c2d8f377-82f8-45c2-9aad-5df49d442e7c","Type":"ContainerDied","Data":"69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a"} Sep 19 09:28:17.659962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.659574 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" Sep 19 09:28:17.659962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.659632 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj" event={"ID":"c2d8f377-82f8-45c2-9aad-5df49d442e7c","Type":"ContainerDied","Data":"2dd00d0a0e47f54a6c8b4060dbd43cb7a9248769a37eba3d1de1e4eb66797169"} Sep 19 09:28:17.659962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.659663 2585 scope.go:117] "RemoveContainer" containerID="69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a" Sep 19 09:28:17.670357 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.670332 2585 scope.go:117] "RemoveContainer" containerID="69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a" Sep 19 09:28:17.670684 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:28:17.670637 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a\": container with ID starting with 69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a not found: ID does not exist" containerID="69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a" Sep 19 09:28:17.670684 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.670666 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a"} err="failed to get container status \"69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a\": rpc error: code = NotFound desc = could not find container \"69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a\": container with ID starting with 69f20996183f5801ce7e5d72eb3104604076b60bc957c7ed5d7d8bfd6a52711a not found: ID does not exist" Sep 19 09:28:17.761637 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.761600 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d8f377-82f8-45c2-9aad-5df49d442e7c-openshift-service-ca\") pod \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " Sep 19 09:28:17.761824 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.761657 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-maas-api-tls\") pod \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " Sep 19 09:28:17.761824 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.761764 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-metrics-certs\") pod \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " Sep 19 09:28:17.761923 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.761831 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkgpg\" (UniqueName: \"kubernetes.io/projected/c2d8f377-82f8-45c2-9aad-5df49d442e7c-kube-api-access-gkgpg\") pod \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\" (UID: \"c2d8f377-82f8-45c2-9aad-5df49d442e7c\") " Sep 19 09:28:17.761986 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.761958 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2d8f377-82f8-45c2-9aad-5df49d442e7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2d8f377-82f8-45c2-9aad-5df49d442e7c" (UID: "c2d8f377-82f8-45c2-9aad-5df49d442e7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:28:17.762196 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.762177 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d8f377-82f8-45c2-9aad-5df49d442e7c-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:17.763882 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.763827 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2d8f377-82f8-45c2-9aad-5df49d442e7c" (UID: "c2d8f377-82f8-45c2-9aad-5df49d442e7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:28:17.763882 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.763842 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d8f377-82f8-45c2-9aad-5df49d442e7c-kube-api-access-gkgpg" (OuterVolumeSpecName: "kube-api-access-gkgpg") pod "c2d8f377-82f8-45c2-9aad-5df49d442e7c" (UID: "c2d8f377-82f8-45c2-9aad-5df49d442e7c"). InnerVolumeSpecName "kube-api-access-gkgpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:28:17.764032 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.763945 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c2d8f377-82f8-45c2-9aad-5df49d442e7c" (UID: "c2d8f377-82f8-45c2-9aad-5df49d442e7c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:28:17.862831 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.862757 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:17.862831 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.862781 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2d8f377-82f8-45c2-9aad-5df49d442e7c-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:17.862831 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.862790 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkgpg\" (UniqueName: \"kubernetes.io/projected/c2d8f377-82f8-45c2-9aad-5df49d442e7c-kube-api-access-gkgpg\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:17.984769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.984737 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj"] Sep 19 09:28:17.988292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:17.988264 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-0ce808-7f5cf567bf-pcxkj"] Sep 19 09:28:19.440849 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:19.440806 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2d8f377-82f8-45c2-9aad-5df49d442e7c" path="/var/lib/kubelet/pods/c2d8f377-82f8-45c2-9aad-5df49d442e7c/volumes" Sep 19 09:28:24.706898 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:24.706817 2585 generic.go:356] "Generic (PLEG): container finished" podID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerID="e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5" exitCode=0 Sep 19 09:28:24.707277 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:24.706890 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" event={"ID":"23da6235-dc15-4820-8f9a-ba0bdf148689","Type":"ContainerDied","Data":"e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5"} Sep 19 09:28:24.707318 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:24.707292 2585 scope.go:117] "RemoveContainer" containerID="e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5" Sep 19 09:28:24.708597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:24.708574 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:28:24.708658 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:24.708642 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:28:24.708701 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:24.708690 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:28:25.717990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:25.717940 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" event={"ID":"23da6235-dc15-4820-8f9a-ba0bdf148689","Type":"ContainerStarted","Data":"66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3"} Sep 19 09:28:26.445545 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:26.444616 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp"] Sep 19 09:28:26.676259 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:26.676226 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:26.676445 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:26.676401 2585 patch_prober.go:28] interesting pod/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Sep 19 09:28:26.676491 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:26.676449 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Sep 19 09:28:26.727004 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:26.726864 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" containerID="cri-o://66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3" gracePeriod=30 Sep 19 09:28:31.550939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:31.550900 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:36.285186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.285161 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:36.356803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356773 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-certs\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.356974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356830 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-data\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.356974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356850 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-envoy\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.356974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356885 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23da6235-dc15-4820-8f9a-ba0bdf148689-istiod-ca-cert\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.356974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356901 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2txk\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-kube-api-access-m2txk\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.356974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356947 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-socket\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.356974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356973 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-token\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.357335 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.356998 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-credential-socket\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.357335 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357033 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-podinfo\") pod \"23da6235-dc15-4820-8f9a-ba0bdf148689\" (UID: \"23da6235-dc15-4820-8f9a-ba0bdf148689\") " Sep 19 09:28:36.357335 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357093 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:28:36.357335 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357256 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-data" (OuterVolumeSpecName: "istio-data") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:28:36.357572 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357368 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23da6235-dc15-4820-8f9a-ba0bdf148689-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:28:36.357572 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357448 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.357572 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357466 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.357741 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357634 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:28:36.357741 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.357696 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:28:36.359162 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.359132 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:28:36.359276 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.359158 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-token" (OuterVolumeSpecName: "istio-token") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:28:36.359324 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.359274 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:28:36.359324 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.359282 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-kube-api-access-m2txk" (OuterVolumeSpecName: "kube-api-access-m2txk") pod "23da6235-dc15-4820-8f9a-ba0bdf148689" (UID: "23da6235-dc15-4820-8f9a-ba0bdf148689"). InnerVolumeSpecName "kube-api-access-m2txk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:28:36.458007 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.457977 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.458007 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.458004 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23da6235-dc15-4820-8f9a-ba0bdf148689-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.458204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.458016 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2txk\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-kube-api-access-m2txk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.458204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.458030 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.458204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.458038 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.458204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.458047 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23da6235-dc15-4820-8f9a-ba0bdf148689-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.458204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.458056 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23da6235-dc15-4820-8f9a-ba0bdf148689-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:28:36.790248 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.790164 2585 generic.go:356] "Generic (PLEG): container finished" podID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerID="66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3" exitCode=0 Sep 19 09:28:36.790248 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.790224 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" event={"ID":"23da6235-dc15-4820-8f9a-ba0bdf148689","Type":"ContainerDied","Data":"66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3"} Sep 19 09:28:36.790248 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.790243 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" Sep 19 09:28:36.790475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.790264 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp" event={"ID":"23da6235-dc15-4820-8f9a-ba0bdf148689","Type":"ContainerDied","Data":"873bc467e0fc6952aa1f355f82617a994d1fc28520460ed6fbb04f2d59dbefd8"} Sep 19 09:28:36.790475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.790287 2585 scope.go:117] "RemoveContainer" containerID="66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3" Sep 19 09:28:36.800403 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.800386 2585 scope.go:117] "RemoveContainer" containerID="e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5" Sep 19 09:28:36.808806 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.808789 2585 scope.go:117] "RemoveContainer" containerID="66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3" Sep 19 09:28:36.809106 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:28:36.809084 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3\": container with ID starting with 66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3 not found: ID does not exist" containerID="66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3" Sep 19 09:28:36.809161 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.809120 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3"} err="failed to get container status \"66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3\": rpc error: code = NotFound desc = could not find container \"66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3\": container with ID starting with 66000f375e37898e9714fcee184582099f5983f918d9621c997932bb498ce3d3 not found: ID does not exist" Sep 19 09:28:36.809161 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.809139 2585 scope.go:117] "RemoveContainer" containerID="e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5" Sep 19 09:28:36.809376 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:28:36.809356 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5\": container with ID starting with e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5 not found: ID does not exist" containerID="e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5" Sep 19 09:28:36.809421 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.809381 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5"} err="failed to get container status \"e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5\": rpc error: code = NotFound desc = could not find container \"e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5\": container with ID starting with e8f4e9efa64c99212e9aca627aa9b685a5002bb5a11f663de4b70d9209185cc5 not found: ID does not exist" Sep 19 09:28:36.815401 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.815376 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp"] Sep 19 09:28:36.821136 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:36.821114 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-61e32580-openshift-default-5b89f8b84-ms5gp"] Sep 19 09:28:37.437540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:28:37.437469 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" path="/var/lib/kubelet/pods/23da6235-dc15-4820-8f9a-ba0bdf148689/volumes" Sep 19 09:29:31.204114 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204081 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q"] Sep 19 09:29:31.204567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204543 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" Sep 19 09:29:31.204567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204557 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" Sep 19 09:29:31.204567 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204568 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" Sep 19 09:29:31.204686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204574 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" Sep 19 09:29:31.204686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204588 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d8f377-82f8-45c2-9aad-5df49d442e7c" containerName="maas-api" Sep 19 09:29:31.204686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204592 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d8f377-82f8-45c2-9aad-5df49d442e7c" containerName="maas-api" Sep 19 09:29:31.204686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204667 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" Sep 19 09:29:31.204686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204675 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="23da6235-dc15-4820-8f9a-ba0bdf148689" containerName="istio-proxy" Sep 19 09:29:31.204686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.204682 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2d8f377-82f8-45c2-9aad-5df49d442e7c" containerName="maas-api" Sep 19 09:29:31.207984 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.207962 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.210561 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.210499 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-ac387c-serving-cert\"" Sep 19 09:29:31.222073 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.222046 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q"] Sep 19 09:29:31.275824 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.275791 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-metrics-certs\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.275995 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.275847 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npsnr\" (UniqueName: \"kubernetes.io/projected/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-kube-api-access-npsnr\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.275995 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.275914 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-openshift-service-ca\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.276084 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.276028 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-maas-api-tls\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.376896 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.376862 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-metrics-certs\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.377062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.376934 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npsnr\" (UniqueName: \"kubernetes.io/projected/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-kube-api-access-npsnr\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.377062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.376963 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-openshift-service-ca\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.377062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.377031 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-maas-api-tls\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.377730 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.377705 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-openshift-service-ca\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.379557 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.379534 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-metrics-certs\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.379733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.379716 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-maas-api-tls\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.386641 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.386622 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npsnr\" (UniqueName: \"kubernetes.io/projected/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-kube-api-access-npsnr\") pod \"maas-api-e2e-explicit-ac387c-69c95f55-kqf4q\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.522993 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.522916 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:31.805844 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.805809 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q"] Sep 19 09:29:31.807021 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:29:31.806985 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad8e9cb_490d_4ec1_aa7b_b3767d2276ce.slice/crio-4e5a89934bd959424704a96205d12dfbac76eea9024c3b7295d3f57a287b5ece WatchSource:0}: Error finding container 4e5a89934bd959424704a96205d12dfbac76eea9024c3b7295d3f57a287b5ece: Status 404 returned error can't find the container with id 4e5a89934bd959424704a96205d12dfbac76eea9024c3b7295d3f57a287b5ece Sep 19 09:29:31.808846 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:31.808828 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:29:32.131312 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.131278 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" event={"ID":"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce","Type":"ContainerStarted","Data":"4e5a89934bd959424704a96205d12dfbac76eea9024c3b7295d3f57a287b5ece"} Sep 19 09:29:32.188218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.188189 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67"] Sep 19 09:29:32.188416 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.188375 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" podUID="9baf916d-70a1-4cab-b1f6-32024558c917" containerName="maas-api" containerID="cri-o://f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724" gracePeriod=30 Sep 19 09:29:32.475327 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.475298 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:29:32.590289 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.590256 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baf916d-70a1-4cab-b1f6-32024558c917-openshift-service-ca\") pod \"9baf916d-70a1-4cab-b1f6-32024558c917\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " Sep 19 09:29:32.590483 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.590307 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls\") pod \"9baf916d-70a1-4cab-b1f6-32024558c917\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " Sep 19 09:29:32.590483 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.590347 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfqqs\" (UniqueName: \"kubernetes.io/projected/9baf916d-70a1-4cab-b1f6-32024558c917-kube-api-access-qfqqs\") pod \"9baf916d-70a1-4cab-b1f6-32024558c917\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " Sep 19 09:29:32.590483 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.590382 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-metrics-certs\") pod \"9baf916d-70a1-4cab-b1f6-32024558c917\" (UID: \"9baf916d-70a1-4cab-b1f6-32024558c917\") " Sep 19 09:29:32.590723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.590699 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9baf916d-70a1-4cab-b1f6-32024558c917-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9baf916d-70a1-4cab-b1f6-32024558c917" (UID: "9baf916d-70a1-4cab-b1f6-32024558c917"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:29:32.592651 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.592620 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9baf916d-70a1-4cab-b1f6-32024558c917" (UID: "9baf916d-70a1-4cab-b1f6-32024558c917"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:29:32.592757 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.592727 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9baf916d-70a1-4cab-b1f6-32024558c917-kube-api-access-qfqqs" (OuterVolumeSpecName: "kube-api-access-qfqqs") pod "9baf916d-70a1-4cab-b1f6-32024558c917" (UID: "9baf916d-70a1-4cab-b1f6-32024558c917"). InnerVolumeSpecName "kube-api-access-qfqqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:29:32.592798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.592771 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9baf916d-70a1-4cab-b1f6-32024558c917" (UID: "9baf916d-70a1-4cab-b1f6-32024558c917"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:29:32.691948 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.691861 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfqqs\" (UniqueName: \"kubernetes.io/projected/9baf916d-70a1-4cab-b1f6-32024558c917-kube-api-access-qfqqs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:29:32.691948 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.691888 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:29:32.691948 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.691897 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baf916d-70a1-4cab-b1f6-32024558c917-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:29:32.691948 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:32.691906 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baf916d-70a1-4cab-b1f6-32024558c917-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:29:33.138386 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.138343 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" event={"ID":"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce","Type":"ContainerStarted","Data":"86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688"} Sep 19 09:29:33.138611 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.138457 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:29:33.139679 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.139652 2585 generic.go:356] "Generic (PLEG): container finished" podID="9baf916d-70a1-4cab-b1f6-32024558c917" containerID="f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724" exitCode=0 Sep 19 09:29:33.139803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.139693 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" Sep 19 09:29:33.139803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.139724 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" event={"ID":"9baf916d-70a1-4cab-b1f6-32024558c917","Type":"ContainerDied","Data":"f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724"} Sep 19 09:29:33.139803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.139764 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67" event={"ID":"9baf916d-70a1-4cab-b1f6-32024558c917","Type":"ContainerDied","Data":"bc316ad5077aea010bd6461680e57514fba063cbf7be8b5625dedfd87535b4d6"} Sep 19 09:29:33.139803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.139785 2585 scope.go:117] "RemoveContainer" containerID="f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724" Sep 19 09:29:33.151183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.151166 2585 scope.go:117] "RemoveContainer" containerID="f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724" Sep 19 09:29:33.151423 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:29:33.151405 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724\": container with ID starting with f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724 not found: ID does not exist" containerID="f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724" Sep 19 09:29:33.151478 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.151429 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724"} err="failed to get container status \"f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724\": rpc error: code = NotFound desc = could not find container \"f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724\": container with ID starting with f8241c6342b4c551f7c7d601c8d2505a5b3f447c8c88fe8d141f23411d9e1724 not found: ID does not exist" Sep 19 09:29:33.167006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.166965 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" podStartSLOduration=1.842777079 podStartE2EDuration="2.166952805s" totalImagesPullingTime="324.175726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:29:33.165151893 +0000 UTC m=+1379.923272920" watchObservedRunningTime="2026-09-19 09:29:33.166952805 +0000 UTC m=+1379.925073879" Sep 19 09:29:33.187371 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.187347 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67"] Sep 19 09:29:33.191528 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.191501 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-acc640-8466649555-6xg67"] Sep 19 09:29:33.436427 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:33.436351 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9baf916d-70a1-4cab-b1f6-32024558c917" path="/var/lib/kubelet/pods/9baf916d-70a1-4cab-b1f6-32024558c917/volumes" Sep 19 09:29:39.152425 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:29:39.152395 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:30:15.294481 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.294443 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz"] Sep 19 09:30:15.294943 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.294884 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9baf916d-70a1-4cab-b1f6-32024558c917" containerName="maas-api" Sep 19 09:30:15.294943 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.294897 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9baf916d-70a1-4cab-b1f6-32024558c917" containerName="maas-api" Sep 19 09:30:15.295022 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.294961 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9baf916d-70a1-4cab-b1f6-32024558c917" containerName="maas-api" Sep 19 09:30:15.298362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.298344 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.301496 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.301472 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0e698776-serving-cert\"" Sep 19 09:30:15.314428 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.314401 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz"] Sep 19 09:30:15.403431 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.403399 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.403612 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.403459 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-metrics-certs\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.403612 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.403510 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-openshift-service-ca\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.403612 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.403552 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klr8x\" (UniqueName: \"kubernetes.io/projected/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-kube-api-access-klr8x\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.504256 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.504219 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.504455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.504301 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-metrics-certs\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.504455 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:30:15.504330 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-serving-cert: secret "maas-api-e2e-mt-0e698776-serving-cert" not found Sep 19 09:30:15.504455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.504355 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-openshift-service-ca\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.504455 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.504386 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klr8x\" (UniqueName: \"kubernetes.io/projected/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-kube-api-access-klr8x\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.504455 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:30:15.504407 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls podName:2999fcef-99d2-4a27-9a1e-2654d70c7a7e nodeName:}" failed. No retries permitted until 2026-09-19 09:30:16.004385089 +0000 UTC m=+1422.762506113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls") pod "maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" (UID: "2999fcef-99d2-4a27-9a1e-2654d70c7a7e") : secret "maas-api-e2e-mt-0e698776-serving-cert" not found Sep 19 09:30:15.505083 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.505053 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-openshift-service-ca\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.507020 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.506996 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-metrics-certs\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:15.522504 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:15.522478 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klr8x\" (UniqueName: \"kubernetes.io/projected/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-kube-api-access-klr8x\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:16.009309 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:16.009275 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:16.012069 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:16.012036 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls\") pod \"maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:16.209346 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:16.209269 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:16.493481 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:16.493441 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz"] Sep 19 09:30:16.494976 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:30:16.494939 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2999fcef_99d2_4a27_9a1e_2654d70c7a7e.slice/crio-edfad064cafbc346419e313fa62e52726d822f4398d29adc63204e891faa4b46 WatchSource:0}: Error finding container edfad064cafbc346419e313fa62e52726d822f4398d29adc63204e891faa4b46: Status 404 returned error can't find the container with id edfad064cafbc346419e313fa62e52726d822f4398d29adc63204e891faa4b46 Sep 19 09:30:17.414671 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:17.414626 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" event={"ID":"2999fcef-99d2-4a27-9a1e-2654d70c7a7e","Type":"ContainerStarted","Data":"e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac"} Sep 19 09:30:17.414671 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:17.414699 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" event={"ID":"2999fcef-99d2-4a27-9a1e-2654d70c7a7e","Type":"ContainerStarted","Data":"edfad064cafbc346419e313fa62e52726d822f4398d29adc63204e891faa4b46"} Sep 19 09:30:17.414671 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:17.414786 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:30:17.433093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:17.433046 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" podStartSLOduration=2.061585459 podStartE2EDuration="2.433026948s" totalImagesPullingTime="371.441489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:30:17.432882908 +0000 UTC m=+1424.191003957" watchObservedRunningTime="2026-09-19 09:30:17.433026948 +0000 UTC m=+1424.191147974" Sep 19 09:30:23.428475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:30:23.428445 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:31:00.286189 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.286157 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz"] Sep 19 09:31:00.286747 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.286372 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" podUID="2999fcef-99d2-4a27-9a1e-2654d70c7a7e" containerName="maas-api" containerID="cri-o://e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac" gracePeriod=30 Sep 19 09:31:00.543884 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.543829 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:31:00.631786 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.631756 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls\") pod \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " Sep 19 09:31:00.631954 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.631807 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-metrics-certs\") pod \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " Sep 19 09:31:00.631954 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.631895 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klr8x\" (UniqueName: \"kubernetes.io/projected/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-kube-api-access-klr8x\") pod \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " Sep 19 09:31:00.631954 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.631931 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-openshift-service-ca\") pod \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\" (UID: \"2999fcef-99d2-4a27-9a1e-2654d70c7a7e\") " Sep 19 09:31:00.632587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.632557 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2999fcef-99d2-4a27-9a1e-2654d70c7a7e" (UID: "2999fcef-99d2-4a27-9a1e-2654d70c7a7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:31:00.635158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.635133 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-kube-api-access-klr8x" (OuterVolumeSpecName: "kube-api-access-klr8x") pod "2999fcef-99d2-4a27-9a1e-2654d70c7a7e" (UID: "2999fcef-99d2-4a27-9a1e-2654d70c7a7e"). InnerVolumeSpecName "kube-api-access-klr8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:31:00.635309 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.635227 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2999fcef-99d2-4a27-9a1e-2654d70c7a7e" (UID: "2999fcef-99d2-4a27-9a1e-2654d70c7a7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:31:00.636508 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.636484 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2999fcef-99d2-4a27-9a1e-2654d70c7a7e" (UID: "2999fcef-99d2-4a27-9a1e-2654d70c7a7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:31:00.671998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.671968 2585 generic.go:356] "Generic (PLEG): container finished" podID="2999fcef-99d2-4a27-9a1e-2654d70c7a7e" containerID="e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac" exitCode=0 Sep 19 09:31:00.672109 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.672079 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" Sep 19 09:31:00.672177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.672100 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" event={"ID":"2999fcef-99d2-4a27-9a1e-2654d70c7a7e","Type":"ContainerDied","Data":"e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac"} Sep 19 09:31:00.672177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.672155 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz" event={"ID":"2999fcef-99d2-4a27-9a1e-2654d70c7a7e","Type":"ContainerDied","Data":"edfad064cafbc346419e313fa62e52726d822f4398d29adc63204e891faa4b46"} Sep 19 09:31:00.672270 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.672178 2585 scope.go:117] "RemoveContainer" containerID="e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac" Sep 19 09:31:00.681649 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.681628 2585 scope.go:117] "RemoveContainer" containerID="e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac" Sep 19 09:31:00.681931 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:31:00.681908 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac\": container with ID starting with e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac not found: ID does not exist" containerID="e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac" Sep 19 09:31:00.681990 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.681939 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac"} err="failed to get container status \"e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac\": rpc error: code = NotFound desc = could not find container \"e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac\": container with ID starting with e11f33899429eefee18f4924c6dcbc96d541dba0d20e3f1d41b3c887aa4c3eac not found: ID does not exist" Sep 19 09:31:00.699754 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.699729 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz"] Sep 19 09:31:00.702850 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.702828 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0e698776-5b9b6c98c-j9prz"] Sep 19 09:31:00.733649 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.733622 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klr8x\" (UniqueName: \"kubernetes.io/projected/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-kube-api-access-klr8x\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:00.733649 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.733650 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:00.733801 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.733667 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:00.733801 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:00.733681 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2999fcef-99d2-4a27-9a1e-2654d70c7a7e-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:01.437290 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:01.437255 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2999fcef-99d2-4a27-9a1e-2654d70c7a7e" path="/var/lib/kubelet/pods/2999fcef-99d2-4a27-9a1e-2654d70c7a7e/volumes" Sep 19 09:31:21.859087 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:21.859049 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc"] Sep 19 09:31:21.859708 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:21.859688 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2999fcef-99d2-4a27-9a1e-2654d70c7a7e" containerName="maas-api" Sep 19 09:31:21.859708 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:21.859711 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2999fcef-99d2-4a27-9a1e-2654d70c7a7e" containerName="maas-api" Sep 19 09:31:21.859845 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:21.859795 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2999fcef-99d2-4a27-9a1e-2654d70c7a7e" containerName="maas-api" Sep 19 09:31:21.864369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:21.864351 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:21.871617 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:21.871594 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-261ce4-openshift-default-dockercfg-hdzsk\"" Sep 19 09:31:21.901063 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:21.901030 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc"] Sep 19 09:31:22.043798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.043760 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-podinfo\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.043798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.043799 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istiod-ca-cert\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.044062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.043828 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-certs\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.044062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.043845 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpl7f\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-kube-api-access-rpl7f\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.044062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.043929 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-socket\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.044062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.043962 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-envoy\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.044062 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.043990 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-token\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.044254 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.044082 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-data\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.044254 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.044144 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-credential-socket\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145080 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.144968 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-socket\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145080 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145022 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-envoy\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145080 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145048 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-token\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145108 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-data\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145168 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-credential-socket\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145237 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-podinfo\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145260 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istiod-ca-cert\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145623 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145429 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-socket\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145623 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145585 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-data\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145739 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145619 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-credential-socket\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145797 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145758 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-certs\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.145918 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145294 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-certs\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.146032 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.145947 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpl7f\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-kube-api-access-rpl7f\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.146082 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.146037 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istiod-ca-cert\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.147364 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.147343 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-envoy\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.147682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.147662 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-podinfo\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.153865 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.153840 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-token\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.155578 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.155560 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpl7f\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-kube-api-access-rpl7f\") pod \"e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.174471 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.174440 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:22.453385 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.453357 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc"] Sep 19 09:31:22.455345 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:31:22.455315 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef36802_3461_41a6_93f5_d8d79d0ccc4d.slice/crio-958ed8cdfc1cdd6018b4dedb872ad9e65f13ee24bfb5440692bd51478ec36029 WatchSource:0}: Error finding container 958ed8cdfc1cdd6018b4dedb872ad9e65f13ee24bfb5440692bd51478ec36029: Status 404 returned error can't find the container with id 958ed8cdfc1cdd6018b4dedb872ad9e65f13ee24bfb5440692bd51478ec36029 Sep 19 09:31:22.457648 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.457611 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:22.457774 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.457720 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:22.457840 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.457770 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:22.802146 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.802059 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" event={"ID":"1ef36802-3461-41a6-93f5-d8d79d0ccc4d","Type":"ContainerStarted","Data":"bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353"} Sep 19 09:31:22.802146 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.802110 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" event={"ID":"1ef36802-3461-41a6-93f5-d8d79d0ccc4d","Type":"ContainerStarted","Data":"958ed8cdfc1cdd6018b4dedb872ad9e65f13ee24bfb5440692bd51478ec36029"} Sep 19 09:31:22.824785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:22.824726 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" podStartSLOduration=1.824707977 podStartE2EDuration="1.824707977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:31:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:31:22.822719853 +0000 UTC m=+1489.580840880" watchObservedRunningTime="2026-09-19 09:31:22.824707977 +0000 UTC m=+1489.582829001" Sep 19 09:31:23.174968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:23.174935 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:23.180426 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:23.180399 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:23.807782 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:23.807747 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:23.808775 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:23.808757 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:35.529158 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:35.529126 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:31:35.533078 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:35.533055 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:31:35.536335 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:35.536304 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:31:35.540844 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:35.540824 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:31:36.381934 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.381900 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q"] Sep 19 09:31:36.382217 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.382170 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" podUID="0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" containerName="maas-api" containerID="cri-o://86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688" gracePeriod=30 Sep 19 09:31:36.636767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.636713 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:31:36.701811 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.701782 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-maas-api-tls\") pod \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " Sep 19 09:31:36.701969 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.701830 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-metrics-certs\") pod \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " Sep 19 09:31:36.701969 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.701953 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-openshift-service-ca\") pod \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " Sep 19 09:31:36.702056 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.701996 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npsnr\" (UniqueName: \"kubernetes.io/projected/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-kube-api-access-npsnr\") pod \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\" (UID: \"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce\") " Sep 19 09:31:36.702296 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.702263 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" (UID: "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:31:36.702408 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.702388 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:36.703926 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.703904 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" (UID: "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:31:36.704017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.703953 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" (UID: "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:31:36.704017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.703972 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-kube-api-access-npsnr" (OuterVolumeSpecName: "kube-api-access-npsnr") pod "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" (UID: "0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce"). InnerVolumeSpecName "kube-api-access-npsnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:31:36.803841 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.803804 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npsnr\" (UniqueName: \"kubernetes.io/projected/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-kube-api-access-npsnr\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:36.803841 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.803842 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:36.804064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.803856 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:31:36.896044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.895955 2585 generic.go:356] "Generic (PLEG): container finished" podID="0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" containerID="86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688" exitCode=0 Sep 19 09:31:36.896044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.895996 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" event={"ID":"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce","Type":"ContainerDied","Data":"86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688"} Sep 19 09:31:36.896044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.896012 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" Sep 19 09:31:36.896044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.896040 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q" event={"ID":"0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce","Type":"ContainerDied","Data":"4e5a89934bd959424704a96205d12dfbac76eea9024c3b7295d3f57a287b5ece"} Sep 19 09:31:36.896355 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.896061 2585 scope.go:117] "RemoveContainer" containerID="86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688" Sep 19 09:31:36.905852 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.905832 2585 scope.go:117] "RemoveContainer" containerID="86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688" Sep 19 09:31:36.906104 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:31:36.906086 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688\": container with ID starting with 86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688 not found: ID does not exist" containerID="86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688" Sep 19 09:31:36.906160 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.906111 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688"} err="failed to get container status \"86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688\": rpc error: code = NotFound desc = could not find container \"86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688\": container with ID starting with 86d3a8003ee8e21f5175c1d4f827616de90d6a4d92c27fd05d9b5c1757e04688 not found: ID does not exist" Sep 19 09:31:36.919570 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.919541 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q"] Sep 19 09:31:36.926320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:36.926298 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ac387c-69c95f55-kqf4q"] Sep 19 09:31:37.436988 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:37.436940 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" path="/var/lib/kubelet/pods/0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce/volumes" Sep 19 09:31:45.052604 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.052500 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6"] Sep 19 09:31:45.053224 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.053202 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" containerName="maas-api" Sep 19 09:31:45.053303 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.053227 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" containerName="maas-api" Sep 19 09:31:45.053725 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.053371 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad8e9cb-490d-4ec1-aa7b-b3767d2276ce" containerName="maas-api" Sep 19 09:31:45.075624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.071342 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.075624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.075472 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-66c0fb-openshift-default-dockercfg-nfztb\"" Sep 19 09:31:45.079495 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.079297 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6"] Sep 19 09:31:45.193111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193072 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-certs\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193116 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-socket\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193135 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-data\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193165 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-podinfo\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193207 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istiod-ca-cert\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193228 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-token\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193254 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-credential-socket\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193295 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-envoy\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.193320 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.193316 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4krz\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-kube-api-access-r4krz\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.293885 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.293849 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-credential-socket\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294182 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294165 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-credential-socket\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294505 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294251 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-envoy\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294574 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294560 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4krz\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-kube-api-access-r4krz\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294621 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294603 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-certs\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294655 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294643 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-socket\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294689 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294670 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-data\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294712 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-podinfo\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294800 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294783 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istiod-ca-cert\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.294842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.294825 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-token\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.295172 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.295146 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-data\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.295358 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.295335 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-certs\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.295439 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.295368 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-socket\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.295610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.295481 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istiod-ca-cert\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.296707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.296683 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-envoy\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.297609 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.297589 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-podinfo\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.302193 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.302162 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4krz\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-kube-api-access-r4krz\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.302475 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.302453 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-token\") pod \"e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.386622 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.386590 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:45.696589 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:31:45.696559 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab42e91_b4b2_4753_ad4f_dcbe2c9b5614.slice/crio-74b8b86e4940fa8f87fa6df6f7c09637dd05789c954e51e42c4a36fdca58b7e1 WatchSource:0}: Error finding container 74b8b86e4940fa8f87fa6df6f7c09637dd05789c954e51e42c4a36fdca58b7e1: Status 404 returned error can't find the container with id 74b8b86e4940fa8f87fa6df6f7c09637dd05789c954e51e42c4a36fdca58b7e1 Sep 19 09:31:45.696870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.696843 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6"] Sep 19 09:31:45.699101 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.699066 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:45.699200 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.699132 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:45.699200 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.699161 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:45.966541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.966418 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" event={"ID":"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614","Type":"ContainerStarted","Data":"2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182"} Sep 19 09:31:45.966541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.966479 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" event={"ID":"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614","Type":"ContainerStarted","Data":"74b8b86e4940fa8f87fa6df6f7c09637dd05789c954e51e42c4a36fdca58b7e1"} Sep 19 09:31:45.989058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:45.989002 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podStartSLOduration=0.988984501 podStartE2EDuration="988.984501ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:31:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:31:45.987542887 +0000 UTC m=+1512.745663911" watchObservedRunningTime="2026-09-19 09:31:45.988984501 +0000 UTC m=+1512.747105526" Sep 19 09:31:46.387600 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:46.387547 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:46.394396 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:46.394367 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:46.973194 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:46.973161 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:46.974271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:46.974249 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:31:48.986790 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:48.986757 2585 generic.go:356] "Generic (PLEG): container finished" podID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerID="bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353" exitCode=0 Sep 19 09:31:48.987221 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:48.986832 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" event={"ID":"1ef36802-3461-41a6-93f5-d8d79d0ccc4d","Type":"ContainerDied","Data":"bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353"} Sep 19 09:31:48.992535 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:48.988846 2585 scope.go:117] "RemoveContainer" containerID="bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353" Sep 19 09:31:48.995191 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:48.995152 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:48.995295 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:48.995230 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:48.995295 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:48.995268 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:31:49.995324 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:49.995278 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" event={"ID":"1ef36802-3461-41a6-93f5-d8d79d0ccc4d","Type":"ContainerStarted","Data":"0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b"} Sep 19 09:31:50.175460 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:50.175426 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:50.176495 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:50.176466 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Sep 19 09:31:50.176663 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:50.176552 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Sep 19 09:31:51.175650 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:51.175611 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Sep 19 09:31:51.176034 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:51.175671 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Sep 19 09:31:52.175604 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:52.175567 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Sep 19 09:31:52.175823 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:52.175624 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Sep 19 09:31:53.175498 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:53.175459 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Sep 19 09:31:53.175676 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:53.175541 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Sep 19 09:31:53.808048 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:53.808016 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:54.175842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:54.175753 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Sep 19 09:31:54.175842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:54.175812 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Sep 19 09:31:55.179668 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:55.179637 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:31:55.180720 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:31:55.180697 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:32:18.014688 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:18.014642 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 19 09:32:18.015185 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:18.014715 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 19 09:32:18.915292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:18.915254 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7845-862df2"] Sep 19 09:32:18.925662 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:18.925631 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:18.929335 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:18.929311 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q5mhq\"" Sep 19 09:32:18.935743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:18.935710 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-862df2"] Sep 19 09:32:19.037715 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.037681 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnnc9\" (UniqueName: \"kubernetes.io/projected/b4594fe0-f070-4998-9eca-d869ad97b7cd-kube-api-access-qnnc9\") pod \"test-curl-7845-862df2\" (UID: \"b4594fe0-f070-4998-9eca-d869ad97b7cd\") " pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:19.139072 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.139032 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnnc9\" (UniqueName: \"kubernetes.io/projected/b4594fe0-f070-4998-9eca-d869ad97b7cd-kube-api-access-qnnc9\") pod \"test-curl-7845-862df2\" (UID: \"b4594fe0-f070-4998-9eca-d869ad97b7cd\") " pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:19.151133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.151095 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnnc9\" (UniqueName: \"kubernetes.io/projected/b4594fe0-f070-4998-9eca-d869ad97b7cd-kube-api-access-qnnc9\") pod \"test-curl-7845-862df2\" (UID: \"b4594fe0-f070-4998-9eca-d869ad97b7cd\") " pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:19.179466 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.179375 2585 generic.go:356] "Generic (PLEG): container finished" podID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerID="2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182" exitCode=0 Sep 19 09:32:19.179466 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.179421 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" event={"ID":"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614","Type":"ContainerDied","Data":"2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182"} Sep 19 09:32:19.179846 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.179829 2585 scope.go:117] "RemoveContainer" containerID="2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182" Sep 19 09:32:19.180956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.180920 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:32:19.181063 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.180998 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:32:19.181063 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.181029 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:32:19.237710 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.237663 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:19.550126 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:19.550094 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-862df2"] Sep 19 09:32:19.551956 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:32:19.551927 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4594fe0_f070_4998_9eca_d869ad97b7cd.slice/crio-0f66e3cbf87fd7bd4af1d0427235d2c9ee7b0ad668ced50e5f731ebf89e860ba WatchSource:0}: Error finding container 0f66e3cbf87fd7bd4af1d0427235d2c9ee7b0ad668ced50e5f731ebf89e860ba: Status 404 returned error can't find the container with id 0f66e3cbf87fd7bd4af1d0427235d2c9ee7b0ad668ced50e5f731ebf89e860ba Sep 19 09:32:20.186628 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:20.186592 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" event={"ID":"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614","Type":"ContainerStarted","Data":"dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa"} Sep 19 09:32:20.187795 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:20.187771 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-862df2" event={"ID":"b4594fe0-f070-4998-9eca-d869ad97b7cd","Type":"ContainerStarted","Data":"0f66e3cbf87fd7bd4af1d0427235d2c9ee7b0ad668ced50e5f731ebf89e860ba"} Sep 19 09:32:20.387102 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:20.387066 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:32:20.387410 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:20.387379 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Sep 19 09:32:20.387559 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:20.387436 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Sep 19 09:32:21.387362 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:21.387327 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Sep 19 09:32:21.387797 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:21.387391 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Sep 19 09:32:22.204210 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:22.204176 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-862df2" event={"ID":"b4594fe0-f070-4998-9eca-d869ad97b7cd","Type":"ContainerStarted","Data":"83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73"} Sep 19 09:32:22.223377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:22.223317 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7845-862df2" podStartSLOduration=1.680255164 podStartE2EDuration="4.223296425s" totalImagesPullingTime="2.543041261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:32:22.218719149 +0000 UTC m=+1548.976840174" watchObservedRunningTime="2026-09-19 09:32:22.223296425 +0000 UTC m=+1548.981417451" Sep 19 09:32:22.387688 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:22.387657 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Sep 19 09:32:22.388104 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:22.387712 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Sep 19 09:32:22.886900 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:22.886860 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7845-862df2"] Sep 19 09:32:22.890899 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:22.890868 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7845-862df2"] Sep 19 09:32:23.001687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.001602 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7845-401d05"] Sep 19 09:32:23.002040 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.002027 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4594fe0-f070-4998-9eca-d869ad97b7cd" containerName="test-curl-7845-862df2" Sep 19 09:32:23.002084 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.002041 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4594fe0-f070-4998-9eca-d869ad97b7cd" containerName="test-curl-7845-862df2" Sep 19 09:32:23.002127 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.002118 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4594fe0-f070-4998-9eca-d869ad97b7cd" containerName="test-curl-7845-862df2" Sep 19 09:32:23.005259 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.005243 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:23.092507 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.092476 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kxrv\" (UniqueName: \"kubernetes.io/projected/811893d8-fef7-4b3b-b659-6008bbe33d8d-kube-api-access-4kxrv\") pod \"test-curl-7845-401d05\" (UID: \"811893d8-fef7-4b3b-b659-6008bbe33d8d\") " pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:23.093274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.093246 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-401d05"] Sep 19 09:32:23.193464 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.193430 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kxrv\" (UniqueName: \"kubernetes.io/projected/811893d8-fef7-4b3b-b659-6008bbe33d8d-kube-api-access-4kxrv\") pod \"test-curl-7845-401d05\" (UID: \"811893d8-fef7-4b3b-b659-6008bbe33d8d\") " pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:23.203683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.203658 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kxrv\" (UniqueName: \"kubernetes.io/projected/811893d8-fef7-4b3b-b659-6008bbe33d8d-kube-api-access-4kxrv\") pod \"test-curl-7845-401d05\" (UID: \"811893d8-fef7-4b3b-b659-6008bbe33d8d\") " pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:23.210069 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:23.210043 2585 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7845-862df2" container="test-curl-7845-862df2" Sep 19 09:32:23.322683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.322651 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:23.387226 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.387198 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Sep 19 09:32:23.387392 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.387361 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Sep 19 09:32:23.608952 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:23.608921 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-401d05"] Sep 19 09:32:23.621790 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:32:23.614631 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod811893d8_fef7_4b3b_b659_6008bbe33d8d.slice/crio-3cfc7205927d68c0747a0c1a7a632b41099141c045a844632cc4a73a467fbfd9 WatchSource:0}: Error finding container 3cfc7205927d68c0747a0c1a7a632b41099141c045a844632cc4a73a467fbfd9: Status 404 returned error can't find the container with id 3cfc7205927d68c0747a0c1a7a632b41099141c045a844632cc4a73a467fbfd9 Sep 19 09:32:24.221561 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.221482 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7845-862df2" podUID="b4594fe0-f070-4998-9eca-d869ad97b7cd" containerName="test-curl-7845-862df2" containerID="cri-o://83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73" gracePeriod=2 Sep 19 09:32:24.221894 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.221865 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-401d05" event={"ID":"811893d8-fef7-4b3b-b659-6008bbe33d8d","Type":"ContainerStarted","Data":"891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca"} Sep 19 09:32:24.222065 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.222047 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-401d05" event={"ID":"811893d8-fef7-4b3b-b659-6008bbe33d8d","Type":"ContainerStarted","Data":"3cfc7205927d68c0747a0c1a7a632b41099141c045a844632cc4a73a467fbfd9"} Sep 19 09:32:24.249028 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.248962 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7845-401d05" podStartSLOduration=2.248931849 podStartE2EDuration="2.248931849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:32:24.245734279 +0000 UTC m=+1551.003855302" watchObservedRunningTime="2026-09-19 09:32:24.248931849 +0000 UTC m=+1551.007052875" Sep 19 09:32:24.387592 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.387554 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Sep 19 09:32:24.387796 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.387623 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Sep 19 09:32:24.888512 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.888481 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7845-401d05"] Sep 19 09:32:24.898143 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:24.898069 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7845-401d05"] Sep 19 09:32:25.031133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.028816 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7845-d5e97c"] Sep 19 09:32:25.031133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.029548 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811893d8-fef7-4b3b-b659-6008bbe33d8d" containerName="test-curl-7845-401d05" Sep 19 09:32:25.031133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.029570 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="811893d8-fef7-4b3b-b659-6008bbe33d8d" containerName="test-curl-7845-401d05" Sep 19 09:32:25.031133 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.029709 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="811893d8-fef7-4b3b-b659-6008bbe33d8d" containerName="test-curl-7845-401d05" Sep 19 09:32:25.033270 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.033244 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:25.044626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.044590 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-d5e97c"] Sep 19 09:32:25.136231 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.136188 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbxmd\" (UniqueName: \"kubernetes.io/projected/12b0aa4d-77f8-4476-8518-668c6bda81e8-kube-api-access-qbxmd\") pod \"test-curl-7845-d5e97c\" (UID: \"12b0aa4d-77f8-4476-8518-668c6bda81e8\") " pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:25.230802 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:25.230703 2585 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7845-401d05" container="test-curl-7845-401d05" Sep 19 09:32:25.237819 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.237775 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbxmd\" (UniqueName: \"kubernetes.io/projected/12b0aa4d-77f8-4476-8518-668c6bda81e8-kube-api-access-qbxmd\") pod \"test-curl-7845-d5e97c\" (UID: \"12b0aa4d-77f8-4476-8518-668c6bda81e8\") " pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:25.246382 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.246350 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbxmd\" (UniqueName: \"kubernetes.io/projected/12b0aa4d-77f8-4476-8518-668c6bda81e8-kube-api-access-qbxmd\") pod \"test-curl-7845-d5e97c\" (UID: \"12b0aa4d-77f8-4476-8518-668c6bda81e8\") " pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:25.349413 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.349379 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:25.387739 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.387704 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Sep 19 09:32:25.387877 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.387772 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Sep 19 09:32:25.652644 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:25.652622 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-d5e97c"] Sep 19 09:32:25.655002 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:32:25.654974 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12b0aa4d_77f8_4476_8518_668c6bda81e8.slice/crio-1b5d73977940a7f38abfc7ccba96e5128fc6698f20eb45094cd29c79037097e7 WatchSource:0}: Error finding container 1b5d73977940a7f38abfc7ccba96e5128fc6698f20eb45094cd29c79037097e7: Status 404 returned error can't find the container with id 1b5d73977940a7f38abfc7ccba96e5128fc6698f20eb45094cd29c79037097e7 Sep 19 09:32:26.238260 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.238225 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-d5e97c" event={"ID":"12b0aa4d-77f8-4476-8518-668c6bda81e8","Type":"ContainerStarted","Data":"3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1"} Sep 19 09:32:26.238675 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.238267 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-d5e97c" event={"ID":"12b0aa4d-77f8-4476-8518-668c6bda81e8","Type":"ContainerStarted","Data":"1b5d73977940a7f38abfc7ccba96e5128fc6698f20eb45094cd29c79037097e7"} Sep 19 09:32:26.238675 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.238426 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7845-401d05" podUID="811893d8-fef7-4b3b-b659-6008bbe33d8d" containerName="test-curl-7845-401d05" containerID="cri-o://891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca" gracePeriod=2 Sep 19 09:32:26.253015 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.252969 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7845-d5e97c" podStartSLOduration=1.252951003 podStartE2EDuration="1.252951003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:32:26.252455718 +0000 UTC m=+1553.010576745" watchObservedRunningTime="2026-09-19 09:32:26.252951003 +0000 UTC m=+1553.011072023" Sep 19 09:32:26.392041 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.392009 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:32:26.392535 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.392471 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:32:26.393551 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.393307 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:32:26.422094 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.422068 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:26.549858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.549766 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnnc9\" (UniqueName: \"kubernetes.io/projected/b4594fe0-f070-4998-9eca-d869ad97b7cd-kube-api-access-qnnc9\") pod \"b4594fe0-f070-4998-9eca-d869ad97b7cd\" (UID: \"b4594fe0-f070-4998-9eca-d869ad97b7cd\") " Sep 19 09:32:26.551977 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.551947 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4594fe0-f070-4998-9eca-d869ad97b7cd-kube-api-access-qnnc9" (OuterVolumeSpecName: "kube-api-access-qnnc9") pod "b4594fe0-f070-4998-9eca-d869ad97b7cd" (UID: "b4594fe0-f070-4998-9eca-d869ad97b7cd"). InnerVolumeSpecName "kube-api-access-qnnc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:32:26.651770 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.651738 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnnc9\" (UniqueName: \"kubernetes.io/projected/b4594fe0-f070-4998-9eca-d869ad97b7cd-kube-api-access-qnnc9\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:32:26.854128 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.854085 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7845-d5e97c"] Sep 19 09:32:26.858130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:26.858096 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7845-d5e97c"] Sep 19 09:32:27.246113 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:27.246010 2585 generic.go:356] "Generic (PLEG): container finished" podID="b4594fe0-f070-4998-9eca-d869ad97b7cd" containerID="83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73" exitCode=137 Sep 19 09:32:27.246113 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:27.246056 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:27.246634 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:27.246123 2585 scope.go:117] "RemoveContainer" containerID="83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73" Sep 19 09:32:27.246634 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:27.246487 2585 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7845-d5e97c" container="test-curl-7845-d5e97c" Sep 19 09:32:27.255650 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:27.255629 2585 scope.go:117] "RemoveContainer" containerID="83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73" Sep 19 09:32:27.256015 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:27.255990 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73\": container with ID starting with 83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73 not found: ID does not exist" containerID="83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73" Sep 19 09:32:27.256112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:27.256024 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73"} err="failed to get container status \"83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73\": rpc error: code = NotFound desc = could not find container \"83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73\": container with ID starting with 83bf7093de11dc6907bee3783f60f170ea9e53e8400f5eaf4b1b8d2adf013c73 not found: ID does not exist" Sep 19 09:32:27.257041 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:27.257012 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7845-862df2\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="b4594fe0-f070-4998-9eca-d869ad97b7cd" pod="opendatahub/test-curl-7845-862df2" Sep 19 09:32:27.436930 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:27.436897 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4594fe0-f070-4998-9eca-d869ad97b7cd" path="/var/lib/kubelet/pods/b4594fe0-f070-4998-9eca-d869ad97b7cd/volumes" Sep 19 09:32:28.254390 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:28.254355 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7845-d5e97c" podUID="12b0aa4d-77f8-4476-8518-668c6bda81e8" containerName="test-curl-7845-d5e97c" containerID="cri-o://3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1" gracePeriod=2 Sep 19 09:32:28.427004 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:28.426975 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:28.573857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:28.573828 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kxrv\" (UniqueName: \"kubernetes.io/projected/811893d8-fef7-4b3b-b659-6008bbe33d8d-kube-api-access-4kxrv\") pod \"811893d8-fef7-4b3b-b659-6008bbe33d8d\" (UID: \"811893d8-fef7-4b3b-b659-6008bbe33d8d\") " Sep 19 09:32:28.575883 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:28.575858 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811893d8-fef7-4b3b-b659-6008bbe33d8d-kube-api-access-4kxrv" (OuterVolumeSpecName: "kube-api-access-4kxrv") pod "811893d8-fef7-4b3b-b659-6008bbe33d8d" (UID: "811893d8-fef7-4b3b-b659-6008bbe33d8d"). InnerVolumeSpecName "kube-api-access-4kxrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:32:28.675536 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:28.675497 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kxrv\" (UniqueName: \"kubernetes.io/projected/811893d8-fef7-4b3b-b659-6008bbe33d8d-kube-api-access-4kxrv\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:32:29.261835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:29.261800 2585 generic.go:356] "Generic (PLEG): container finished" podID="811893d8-fef7-4b3b-b659-6008bbe33d8d" containerID="891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca" exitCode=137 Sep 19 09:32:29.262247 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:29.261847 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:29.262247 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:29.261899 2585 scope.go:117] "RemoveContainer" containerID="891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca" Sep 19 09:32:29.270942 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:29.270915 2585 scope.go:117] "RemoveContainer" containerID="891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca" Sep 19 09:32:29.271266 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:29.271242 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca\": container with ID starting with 891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca not found: ID does not exist" containerID="891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca" Sep 19 09:32:29.271355 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:29.271276 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca"} err="failed to get container status \"891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca\": rpc error: code = NotFound desc = could not find container \"891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca\": container with ID starting with 891fb67bedeff82890353c8fa9bff2d38311698452b8cb68de4b20b8261ee0ca not found: ID does not exist" Sep 19 09:32:29.272488 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:29.272457 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7845-401d05\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="811893d8-fef7-4b3b-b659-6008bbe33d8d" pod="opendatahub/test-curl-7845-401d05" Sep 19 09:32:29.440747 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:29.440693 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="811893d8-fef7-4b3b-b659-6008bbe33d8d" path="/var/lib/kubelet/pods/811893d8-fef7-4b3b-b659-6008bbe33d8d/volumes" Sep 19 09:32:30.442647 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:30.442618 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:30.594858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:30.594820 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbxmd\" (UniqueName: \"kubernetes.io/projected/12b0aa4d-77f8-4476-8518-668c6bda81e8-kube-api-access-qbxmd\") pod \"12b0aa4d-77f8-4476-8518-668c6bda81e8\" (UID: \"12b0aa4d-77f8-4476-8518-668c6bda81e8\") " Sep 19 09:32:30.596973 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:30.596943 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12b0aa4d-77f8-4476-8518-668c6bda81e8-kube-api-access-qbxmd" (OuterVolumeSpecName: "kube-api-access-qbxmd") pod "12b0aa4d-77f8-4476-8518-668c6bda81e8" (UID: "12b0aa4d-77f8-4476-8518-668c6bda81e8"). InnerVolumeSpecName "kube-api-access-qbxmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:32:30.695833 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:30.695796 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbxmd\" (UniqueName: \"kubernetes.io/projected/12b0aa4d-77f8-4476-8518-668c6bda81e8-kube-api-access-qbxmd\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:32:31.277649 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:31.277612 2585 generic.go:356] "Generic (PLEG): container finished" podID="12b0aa4d-77f8-4476-8518-668c6bda81e8" containerID="3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1" exitCode=137 Sep 19 09:32:31.277840 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:31.277652 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:31.277840 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:31.277686 2585 scope.go:117] "RemoveContainer" containerID="3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1" Sep 19 09:32:31.287280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:31.287265 2585 scope.go:117] "RemoveContainer" containerID="3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1" Sep 19 09:32:31.287567 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:31.287545 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1\": container with ID starting with 3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1 not found: ID does not exist" containerID="3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1" Sep 19 09:32:31.287648 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:31.287573 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1"} err="failed to get container status \"3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1\": rpc error: code = NotFound desc = could not find container \"3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1\": container with ID starting with 3690dc7c0c33110ef0508d1fbb3ee929e3691381be6c4759f21f5bedbd62f9a1 not found: ID does not exist" Sep 19 09:32:31.288243 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:31.288226 2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7845-d5e97c\" is forbidden: User \"system:node:ip-10-0-132-57.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-57.ec2.internal' and this object" podUID="12b0aa4d-77f8-4476-8518-668c6bda81e8" pod="opendatahub/test-curl-7845-d5e97c" Sep 19 09:32:31.436929 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:31.436892 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12b0aa4d-77f8-4476-8518-668c6bda81e8" path="/var/lib/kubelet/pods/12b0aa4d-77f8-4476-8518-668c6bda81e8/volumes" Sep 19 09:32:37.377574 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.377511 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt"] Sep 19 09:32:37.378195 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.378174 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12b0aa4d-77f8-4476-8518-668c6bda81e8" containerName="test-curl-7845-d5e97c" Sep 19 09:32:37.378332 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.378197 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="12b0aa4d-77f8-4476-8518-668c6bda81e8" containerName="test-curl-7845-d5e97c" Sep 19 09:32:37.378332 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.378294 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="12b0aa4d-77f8-4476-8518-668c6bda81e8" containerName="test-curl-7845-d5e97c" Sep 19 09:32:37.382960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.382937 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.385721 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.385600 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e46994f4-serving-cert\"" Sep 19 09:32:37.397601 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.397578 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt"] Sep 19 09:32:37.563965 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.563929 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.564184 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.564081 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-openshift-service-ca\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.564184 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.564146 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvzvk\" (UniqueName: \"kubernetes.io/projected/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-kube-api-access-fvzvk\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.564184 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.564179 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-metrics-certs\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.665747 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.665648 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.665923 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.665754 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-openshift-service-ca\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.665923 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.665788 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvzvk\" (UniqueName: \"kubernetes.io/projected/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-kube-api-access-fvzvk\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.665923 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:37.665806 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-serving-cert: secret "maas-api-e2e-mt-e46994f4-serving-cert" not found Sep 19 09:32:37.665923 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:32:37.665887 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls podName:f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a nodeName:}" failed. No retries permitted until 2026-09-19 09:32:38.165868561 +0000 UTC m=+1564.923989583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls") pod "maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" (UID: "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a") : secret "maas-api-e2e-mt-e46994f4-serving-cert" not found Sep 19 09:32:37.665923 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.665811 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-metrics-certs\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.666384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.666365 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-openshift-service-ca\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.668368 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.668344 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-metrics-certs\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:37.685285 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:37.685261 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvzvk\" (UniqueName: \"kubernetes.io/projected/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-kube-api-access-fvzvk\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:38.172342 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:38.172309 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:38.185307 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:38.178228 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls\") pod \"maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:38.294233 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:38.294196 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:38.572300 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:38.572271 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt"] Sep 19 09:32:38.573966 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:32:38.573941 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf294dee3_40a0_4d3c_aa8b_eba68c1f0c2a.slice/crio-ae05efbdc67087edfd9beb703fcd319033566cc3f995aa72bad8e4f880c31cfe WatchSource:0}: Error finding container ae05efbdc67087edfd9beb703fcd319033566cc3f995aa72bad8e4f880c31cfe: Status 404 returned error can't find the container with id ae05efbdc67087edfd9beb703fcd319033566cc3f995aa72bad8e4f880c31cfe Sep 19 09:32:39.332880 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:39.332794 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" event={"ID":"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a","Type":"ContainerStarted","Data":"38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23"} Sep 19 09:32:39.332880 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:39.332844 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" event={"ID":"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a","Type":"ContainerStarted","Data":"ae05efbdc67087edfd9beb703fcd319033566cc3f995aa72bad8e4f880c31cfe"} Sep 19 09:32:39.333093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:39.332946 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:32:39.355935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:39.355889 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" podStartSLOduration=1.85056144 podStartE2EDuration="2.355873731s" totalImagesPullingTime="505.312291ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:32:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:32:39.354554476 +0000 UTC m=+1566.112675494" watchObservedRunningTime="2026-09-19 09:32:39.355873731 +0000 UTC m=+1566.113994756" Sep 19 09:32:45.345930 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:32:45.345899 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:33:25.301176 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.301139 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw"] Sep 19 09:33:25.301891 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.301393 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" podUID="aee54331-9423-4274-83e5-2320e67245a8" containerName="maas-api" containerID="cri-o://e9a7f9ab284c1934427901c3dbbf935036c8a9ac1b138fee3b1c45485b859a83" gracePeriod=30 Sep 19 09:33:25.364234 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.364204 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58"] Sep 19 09:33:25.364533 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.364477 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" podUID="e11cc70b-3ed3-412d-bd16-9e646d3c4a52" containerName="payload-processing" containerID="cri-o://c2bc8c2574862e7c229d688be1a587257b7abb02245bde3e5eb9e16eed011db1" gracePeriod=30 Sep 19 09:33:25.611348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.611315 2585 generic.go:356] "Generic (PLEG): container finished" podID="e11cc70b-3ed3-412d-bd16-9e646d3c4a52" containerID="c2bc8c2574862e7c229d688be1a587257b7abb02245bde3e5eb9e16eed011db1" exitCode=0 Sep 19 09:33:25.611563 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.611451 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" event={"ID":"e11cc70b-3ed3-412d-bd16-9e646d3c4a52","Type":"ContainerDied","Data":"c2bc8c2574862e7c229d688be1a587257b7abb02245bde3e5eb9e16eed011db1"} Sep 19 09:33:25.613380 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.613353 2585 generic.go:356] "Generic (PLEG): container finished" podID="aee54331-9423-4274-83e5-2320e67245a8" containerID="e9a7f9ab284c1934427901c3dbbf935036c8a9ac1b138fee3b1c45485b859a83" exitCode=0 Sep 19 09:33:25.613502 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.613431 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" event={"ID":"aee54331-9423-4274-83e5-2320e67245a8","Type":"ContainerDied","Data":"e9a7f9ab284c1934427901c3dbbf935036c8a9ac1b138fee3b1c45485b859a83"} Sep 19 09:33:25.613502 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.613487 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" event={"ID":"aee54331-9423-4274-83e5-2320e67245a8","Type":"ContainerDied","Data":"633eefa11dd187f39065eafd69d1605a36b97011b18f45dc66279b7db3572f8d"} Sep 19 09:33:25.613704 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.613678 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="633eefa11dd187f39065eafd69d1605a36b97011b18f45dc66279b7db3572f8d" Sep 19 09:33:25.630039 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.630010 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:33:25.639717 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.639680 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls\") pod \"aee54331-9423-4274-83e5-2320e67245a8\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " Sep 19 09:33:25.639880 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.639737 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee54331-9423-4274-83e5-2320e67245a8-openshift-service-ca\") pod \"aee54331-9423-4274-83e5-2320e67245a8\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " Sep 19 09:33:25.639880 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.639790 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-metrics-certs\") pod \"aee54331-9423-4274-83e5-2320e67245a8\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " Sep 19 09:33:25.639880 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.639827 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6ljz\" (UniqueName: \"kubernetes.io/projected/aee54331-9423-4274-83e5-2320e67245a8-kube-api-access-j6ljz\") pod \"aee54331-9423-4274-83e5-2320e67245a8\" (UID: \"aee54331-9423-4274-83e5-2320e67245a8\") " Sep 19 09:33:25.640322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.640292 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee54331-9423-4274-83e5-2320e67245a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aee54331-9423-4274-83e5-2320e67245a8" (UID: "aee54331-9423-4274-83e5-2320e67245a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:33:25.643280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.643243 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aee54331-9423-4274-83e5-2320e67245a8" (UID: "aee54331-9423-4274-83e5-2320e67245a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:33:25.643280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.643268 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee54331-9423-4274-83e5-2320e67245a8-kube-api-access-j6ljz" (OuterVolumeSpecName: "kube-api-access-j6ljz") pod "aee54331-9423-4274-83e5-2320e67245a8" (UID: "aee54331-9423-4274-83e5-2320e67245a8"). InnerVolumeSpecName "kube-api-access-j6ljz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:33:25.643896 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.643863 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aee54331-9423-4274-83e5-2320e67245a8" (UID: "aee54331-9423-4274-83e5-2320e67245a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:33:25.685849 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.685818 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:33:25.741034 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.740998 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9s2f\" (UniqueName: \"kubernetes.io/projected/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-kube-api-access-s9s2f\") pod \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " Sep 19 09:33:25.741205 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.741084 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-plugins-config-volume\") pod \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\" (UID: \"e11cc70b-3ed3-412d-bd16-9e646d3c4a52\") " Sep 19 09:33:25.741452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.741428 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e11cc70b-3ed3-412d-bd16-9e646d3c4a52" (UID: "e11cc70b-3ed3-412d-bd16-9e646d3c4a52"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:33:25.741626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.741582 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:25.741626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.741605 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aee54331-9423-4274-83e5-2320e67245a8-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:25.741626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.741620 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aee54331-9423-4274-83e5-2320e67245a8-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:25.741736 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.741633 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6ljz\" (UniqueName: \"kubernetes.io/projected/aee54331-9423-4274-83e5-2320e67245a8-kube-api-access-j6ljz\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:25.741736 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.741646 2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-plugins-config-volume\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:25.743184 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.743161 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-kube-api-access-s9s2f" (OuterVolumeSpecName: "kube-api-access-s9s2f") pod "e11cc70b-3ed3-412d-bd16-9e646d3c4a52" (UID: "e11cc70b-3ed3-412d-bd16-9e646d3c4a52"). InnerVolumeSpecName "kube-api-access-s9s2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:33:25.842967 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:25.842878 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9s2f\" (UniqueName: \"kubernetes.io/projected/e11cc70b-3ed3-412d-bd16-9e646d3c4a52-kube-api-access-s9s2f\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:26.621880 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.621797 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" Sep 19 09:33:26.621880 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.621839 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58" event={"ID":"e11cc70b-3ed3-412d-bd16-9e646d3c4a52","Type":"ContainerDied","Data":"805736656ba5d1f1a9b0a52f0d09a504158155c696107a32851cb28afd1c21f8"} Sep 19 09:33:26.622366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.621912 2585 scope.go:117] "RemoveContainer" containerID="c2bc8c2574862e7c229d688be1a587257b7abb02245bde3e5eb9e16eed011db1" Sep 19 09:33:26.622366 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.622029 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw" Sep 19 09:33:26.649141 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.649114 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw"] Sep 19 09:33:26.654851 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.654824 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9dfb0f-59c47fc98-fd4dw"] Sep 19 09:33:26.667777 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.667751 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58"] Sep 19 09:33:26.671973 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:26.671949 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9dfb0f-85ffc998cf-5bt58"] Sep 19 09:33:27.438064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:27.438031 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee54331-9423-4274-83e5-2320e67245a8" path="/var/lib/kubelet/pods/aee54331-9423-4274-83e5-2320e67245a8/volumes" Sep 19 09:33:27.438417 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:27.438403 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e11cc70b-3ed3-412d-bd16-9e646d3c4a52" path="/var/lib/kubelet/pods/e11cc70b-3ed3-412d-bd16-9e646d3c4a52/volumes" Sep 19 09:33:32.738959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:32.738925 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr"] Sep 19 09:33:32.739330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:32.739153 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" containerID="cri-o://2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4" gracePeriod=30 Sep 19 09:33:35.622878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:35.622850 2585 scope.go:117] "RemoveContainer" containerID="e9a7f9ab284c1934427901c3dbbf935036c8a9ac1b138fee3b1c45485b859a83" Sep 19 09:33:37.998243 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:37.998187 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:33:38.063419 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063388 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38151ef5-5a51-4648-b24c-4701ad991d4d-istiod-ca-cert\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063419 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063423 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-token\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063458 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-data\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063479 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-envoy\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063558 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js949\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-kube-api-access-js949\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063602 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-socket\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063652 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-credential-socket\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063740 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-certs\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063785 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-podinfo\") pod \"38151ef5-5a51-4648-b24c-4701ad991d4d\" (UID: \"38151ef5-5a51-4648-b24c-4701ad991d4d\") " Sep 19 09:33:38.063968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063801 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38151ef5-5a51-4648-b24c-4701ad991d4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:33:38.063968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063801 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:38.063968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063904 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:38.064198 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.063996 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:38.064198 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.064032 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:38.064282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.064192 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.064282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.064218 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.064282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.064231 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.064282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.064244 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.064282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.064257 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38151ef5-5a51-4648-b24c-4701ad991d4d-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.065955 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.065929 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:38.065955 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.065933 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-kube-api-access-js949" (OuterVolumeSpecName: "kube-api-access-js949") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "kube-api-access-js949". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:33:38.066146 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.066128 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:33:38.066185 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.066151 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "38151ef5-5a51-4648-b24c-4701ad991d4d" (UID: "38151ef5-5a51-4648-b24c-4701ad991d4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:33:38.164926 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.164896 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.164926 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.164918 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.164926 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.164934 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38151ef5-5a51-4648-b24c-4701ad991d4d-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.165188 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.164943 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js949\" (UniqueName: \"kubernetes.io/projected/38151ef5-5a51-4648-b24c-4701ad991d4d-kube-api-access-js949\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:38.699891 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.699857 2585 generic.go:356] "Generic (PLEG): container finished" podID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerID="2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4" exitCode=0 Sep 19 09:33:38.700090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.699904 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" Sep 19 09:33:38.700090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.699932 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" event={"ID":"38151ef5-5a51-4648-b24c-4701ad991d4d","Type":"ContainerDied","Data":"2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4"} Sep 19 09:33:38.700090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.699985 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr" event={"ID":"38151ef5-5a51-4648-b24c-4701ad991d4d","Type":"ContainerDied","Data":"a57a6d9d09aedb2ca3451f44f15892eda2b268fbec4652d8c6d0dfb30ecb7397"} Sep 19 09:33:38.700090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.700009 2585 scope.go:117] "RemoveContainer" containerID="2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4" Sep 19 09:33:38.709935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.709919 2585 scope.go:117] "RemoveContainer" containerID="942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359" Sep 19 09:33:38.719956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.719937 2585 scope.go:117] "RemoveContainer" containerID="2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4" Sep 19 09:33:38.720213 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:33:38.720195 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4\": container with ID starting with 2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4 not found: ID does not exist" containerID="2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4" Sep 19 09:33:38.720256 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.720221 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4"} err="failed to get container status \"2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4\": rpc error: code = NotFound desc = could not find container \"2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4\": container with ID starting with 2576fa5b5f0f55b81e0c5c37deb0989abd2e681a3388607643199892502753b4 not found: ID does not exist" Sep 19 09:33:38.720256 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.720237 2585 scope.go:117] "RemoveContainer" containerID="942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359" Sep 19 09:33:38.720505 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:33:38.720482 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359\": container with ID starting with 942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359 not found: ID does not exist" containerID="942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359" Sep 19 09:33:38.720591 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.720515 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359"} err="failed to get container status \"942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359\": rpc error: code = NotFound desc = could not find container \"942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359\": container with ID starting with 942037db7f60fc81c7db2c42e03d1e0338ee8b9ddcab1a89094d608515fd8359 not found: ID does not exist" Sep 19 09:33:38.722873 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.722852 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr"] Sep 19 09:33:38.725187 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:38.725167 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-9dfb0f-openshift-default-67f9d67656-wfstr"] Sep 19 09:33:39.437068 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:39.437038 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" path="/var/lib/kubelet/pods/38151ef5-5a51-4648-b24c-4701ad991d4d/volumes" Sep 19 09:33:43.574072 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.574037 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv"] Sep 19 09:33:43.574541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.574409 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" podUID="9b2591f3-cdf3-49e3-a269-c8644d908a60" containerName="maas-api" containerID="cri-o://866bce3357cbca4c89251ebe98c1f3721f66b6676ab77a337831284bc6b6889f" gracePeriod=30 Sep 19 09:33:43.733741 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.733709 2585 generic.go:356] "Generic (PLEG): container finished" podID="9b2591f3-cdf3-49e3-a269-c8644d908a60" containerID="866bce3357cbca4c89251ebe98c1f3721f66b6676ab77a337831284bc6b6889f" exitCode=0 Sep 19 09:33:43.733895 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.733786 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" event={"ID":"9b2591f3-cdf3-49e3-a269-c8644d908a60","Type":"ContainerDied","Data":"866bce3357cbca4c89251ebe98c1f3721f66b6676ab77a337831284bc6b6889f"} Sep 19 09:33:43.836970 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.836906 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:33:43.915345 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.915314 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2591f3-cdf3-49e3-a269-c8644d908a60-openshift-service-ca\") pod \"9b2591f3-cdf3-49e3-a269-c8644d908a60\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " Sep 19 09:33:43.915565 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.915415 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-metrics-certs\") pod \"9b2591f3-cdf3-49e3-a269-c8644d908a60\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " Sep 19 09:33:43.915565 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.915483 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls\") pod \"9b2591f3-cdf3-49e3-a269-c8644d908a60\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " Sep 19 09:33:43.915565 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.915555 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj8sb\" (UniqueName: \"kubernetes.io/projected/9b2591f3-cdf3-49e3-a269-c8644d908a60-kube-api-access-bj8sb\") pod \"9b2591f3-cdf3-49e3-a269-c8644d908a60\" (UID: \"9b2591f3-cdf3-49e3-a269-c8644d908a60\") " Sep 19 09:33:43.915768 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.915735 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b2591f3-cdf3-49e3-a269-c8644d908a60-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b2591f3-cdf3-49e3-a269-c8644d908a60" (UID: "9b2591f3-cdf3-49e3-a269-c8644d908a60"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:33:43.915936 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.915918 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b2591f3-cdf3-49e3-a269-c8644d908a60-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:43.918224 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.918198 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b2591f3-cdf3-49e3-a269-c8644d908a60" (UID: "9b2591f3-cdf3-49e3-a269-c8644d908a60"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:33:43.918347 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.918261 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b2591f3-cdf3-49e3-a269-c8644d908a60-kube-api-access-bj8sb" (OuterVolumeSpecName: "kube-api-access-bj8sb") pod "9b2591f3-cdf3-49e3-a269-c8644d908a60" (UID: "9b2591f3-cdf3-49e3-a269-c8644d908a60"). InnerVolumeSpecName "kube-api-access-bj8sb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:33:43.918474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:43.918455 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9b2591f3-cdf3-49e3-a269-c8644d908a60" (UID: "9b2591f3-cdf3-49e3-a269-c8644d908a60"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:33:44.017222 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.017184 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:44.017222 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.017213 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj8sb\" (UniqueName: \"kubernetes.io/projected/9b2591f3-cdf3-49e3-a269-c8644d908a60-kube-api-access-bj8sb\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:44.017222 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.017226 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b2591f3-cdf3-49e3-a269-c8644d908a60-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:44.741354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.741324 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" Sep 19 09:33:44.741840 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.741368 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv" event={"ID":"9b2591f3-cdf3-49e3-a269-c8644d908a60","Type":"ContainerDied","Data":"6a8c59bf174c984e189f9c35f526d3aed762bf2cc6125f1d6f5401e0b8fb8f34"} Sep 19 09:33:44.741840 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.741434 2585 scope.go:117] "RemoveContainer" containerID="866bce3357cbca4c89251ebe98c1f3721f66b6676ab77a337831284bc6b6889f" Sep 19 09:33:44.767014 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.766989 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv"] Sep 19 09:33:44.773540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:44.773509 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c18110-679fcdb68d-rm7hv"] Sep 19 09:33:45.436202 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:45.436173 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b2591f3-cdf3-49e3-a269-c8644d908a60" path="/var/lib/kubelet/pods/9b2591f3-cdf3-49e3-a269-c8644d908a60/volumes" Sep 19 09:33:50.206679 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:50.206644 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx"] Sep 19 09:33:50.207074 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:50.206871 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" containerID="cri-o://267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d" gracePeriod=30 Sep 19 09:33:51.022100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022069 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b"] Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022583 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022604 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022618 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e11cc70b-3ed3-412d-bd16-9e646d3c4a52" containerName="payload-processing" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022631 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e11cc70b-3ed3-412d-bd16-9e646d3c4a52" containerName="payload-processing" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022656 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee54331-9423-4274-83e5-2320e67245a8" containerName="maas-api" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022664 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee54331-9423-4274-83e5-2320e67245a8" containerName="maas-api" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022683 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b2591f3-cdf3-49e3-a269-c8644d908a60" containerName="maas-api" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022691 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b2591f3-cdf3-49e3-a269-c8644d908a60" containerName="maas-api" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022789 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b2591f3-cdf3-49e3-a269-c8644d908a60" containerName="maas-api" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022800 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022810 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022822 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e11cc70b-3ed3-412d-bd16-9e646d3c4a52" containerName="payload-processing" Sep 19 09:33:51.022909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.022833 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee54331-9423-4274-83e5-2320e67245a8" containerName="maas-api" Sep 19 09:33:51.028257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.027846 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.030474 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.030454 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-3240ec-openshift-default-dockercfg-xdw9z\"" Sep 19 09:33:51.039925 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.039890 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b"] Sep 19 09:33:51.189317 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-credential-socket\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189372 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-certs\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189406 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxblt\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-kube-api-access-fxblt\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189427 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-data\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189445 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-podinfo\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189709 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189617 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-socket\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189709 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189674 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-envoy\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189759 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istiod-ca-cert\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.189838 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.189788 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-token\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.290835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.290730 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istiod-ca-cert\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.290835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.290788 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-token\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.290835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.290817 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-credential-socket\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.290850 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-certs\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.290874 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxblt\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-kube-api-access-fxblt\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291019 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-data\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291064 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-podinfo\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291126 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-socket\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291176 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-envoy\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291278 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-certs\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291338 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-credential-socket\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291444 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291377 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-data\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291897 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291574 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-socket\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.291897 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.291676 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istiod-ca-cert\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.293918 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.293897 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-envoy\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.294215 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.294192 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-podinfo\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.299077 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.299053 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxblt\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-kube-api-access-fxblt\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.299185 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.299086 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-token\") pod \"e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.341308 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.341268 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:51.623973 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.623938 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b"] Sep 19 09:33:51.626619 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:33:51.626581 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ccb79d_46dc_4ff9_8a99_15d7112ddcdc.slice/crio-5d35be9b11399acbf01ec7a8d5c23dca8f0a56a265cd6e850fd59de4a963dd6b WatchSource:0}: Error finding container 5d35be9b11399acbf01ec7a8d5c23dca8f0a56a265cd6e850fd59de4a963dd6b: Status 404 returned error can't find the container with id 5d35be9b11399acbf01ec7a8d5c23dca8f0a56a265cd6e850fd59de4a963dd6b Sep 19 09:33:51.629010 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.628976 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:33:51.629100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.629044 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:33:51.629100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.629070 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:33:51.785263 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.785222 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" event={"ID":"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc","Type":"ContainerStarted","Data":"53a5bbcf2c219166a6896b9e8c2fdfc7cf030e680212fb8c1c1b175b784e3dc2"} Sep 19 09:33:51.785448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.785286 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" event={"ID":"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc","Type":"ContainerStarted","Data":"5d35be9b11399acbf01ec7a8d5c23dca8f0a56a265cd6e850fd59de4a963dd6b"} Sep 19 09:33:51.808739 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:51.808690 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podStartSLOduration=1.808672526 podStartE2EDuration="1.808672526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:33:51.808152292 +0000 UTC m=+1638.566273317" watchObservedRunningTime="2026-09-19 09:33:51.808672526 +0000 UTC m=+1638.566793551" Sep 19 09:33:52.343343 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:52.343308 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:52.350562 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:52.350532 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:52.790661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:52.790577 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:52.791610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:52.791591 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:33:53.481662 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:53.481627 2585 patch_prober.go:28] interesting pod/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 19 09:33:53.482057 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:53.481682 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 09:33:55.464633 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.464611 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:33:55.638218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638184 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-envoy\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638223 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-data\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638299 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-podinfo\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638324 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-certs\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638347 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-credential-socket\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638448 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638384 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-socket\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638702 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638448 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-token\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638702 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638478 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpmh7\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-kube-api-access-tpmh7\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638702 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638503 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istiod-ca-cert\") pod \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\" (UID: \"d36ec3b2-5604-468e-8cec-f5b4160cfa09\") " Sep 19 09:33:55.638702 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638544 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-data" (OuterVolumeSpecName: "istio-data") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:55.638702 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638638 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:55.638702 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638652 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:55.638991 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638799 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:55.638991 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.638885 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:33:55.639099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.639042 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.639099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.639063 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.639099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.639077 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.639099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.639089 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.639292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.639101 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.640796 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.640768 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:33:55.640796 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.640786 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-kube-api-access-tpmh7" (OuterVolumeSpecName: "kube-api-access-tpmh7") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "kube-api-access-tpmh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:33:55.641053 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.641037 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:33:55.641097 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.641037 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-token" (OuterVolumeSpecName: "istio-token") pod "d36ec3b2-5604-468e-8cec-f5b4160cfa09" (UID: "d36ec3b2-5604-468e-8cec-f5b4160cfa09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:33:55.740541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.740495 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.740541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.740543 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.740729 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.740552 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpmh7\" (UniqueName: \"kubernetes.io/projected/d36ec3b2-5604-468e-8cec-f5b4160cfa09-kube-api-access-tpmh7\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.740729 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.740561 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d36ec3b2-5604-468e-8cec-f5b4160cfa09-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:33:55.810039 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.810011 2585 generic.go:356] "Generic (PLEG): container finished" podID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerID="267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d" exitCode=0 Sep 19 09:33:55.810183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.810062 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" Sep 19 09:33:55.810183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.810090 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" event={"ID":"d36ec3b2-5604-468e-8cec-f5b4160cfa09","Type":"ContainerDied","Data":"267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d"} Sep 19 09:33:55.810183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.810145 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx" event={"ID":"d36ec3b2-5604-468e-8cec-f5b4160cfa09","Type":"ContainerDied","Data":"9975d0ecf90a1c598d123ccdb13415471795528572891391fcf46a0d54798be3"} Sep 19 09:33:55.810183 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.810167 2585 scope.go:117] "RemoveContainer" containerID="267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d" Sep 19 09:33:55.820762 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.820744 2585 scope.go:117] "RemoveContainer" containerID="5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555" Sep 19 09:33:55.829613 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.829591 2585 scope.go:117] "RemoveContainer" containerID="267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d" Sep 19 09:33:55.829894 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:33:55.829875 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d\": container with ID starting with 267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d not found: ID does not exist" containerID="267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d" Sep 19 09:33:55.829939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.829906 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d"} err="failed to get container status \"267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d\": rpc error: code = NotFound desc = could not find container \"267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d\": container with ID starting with 267efb015a90bc912e155eba8744626422cdb22e2aff80baedfae6e8faf7179d not found: ID does not exist" Sep 19 09:33:55.829939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.829923 2585 scope.go:117] "RemoveContainer" containerID="5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555" Sep 19 09:33:55.830158 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:33:55.830143 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555\": container with ID starting with 5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555 not found: ID does not exist" containerID="5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555" Sep 19 09:33:55.830197 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.830161 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555"} err="failed to get container status \"5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555\": rpc error: code = NotFound desc = could not find container \"5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555\": container with ID starting with 5ff8735cc33b906a05f16efb42b00725ae72def79f1510470fe55bc94ca75555 not found: ID does not exist" Sep 19 09:33:55.833762 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.833742 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx"] Sep 19 09:33:55.837650 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:55.837628 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c18110-openshift-default-6f86595db8-5hjsx"] Sep 19 09:33:57.372731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.372678 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449"] Sep 19 09:33:57.373363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.373346 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" Sep 19 09:33:57.373466 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.373367 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" Sep 19 09:33:57.373466 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.373387 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" Sep 19 09:33:57.373466 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.373396 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="38151ef5-5a51-4648-b24c-4701ad991d4d" containerName="istio-proxy" Sep 19 09:33:57.373704 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.373501 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" Sep 19 09:33:57.373828 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.373798 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" Sep 19 09:33:57.379311 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.379289 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.382909 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.381747 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-3240ec-serving-cert\"" Sep 19 09:33:57.385686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.385660 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449"] Sep 19 09:33:57.437022 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.436989 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" path="/var/lib/kubelet/pods/d36ec3b2-5604-468e-8cec-f5b4160cfa09/volumes" Sep 19 09:33:57.559480 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.559443 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.559683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.559486 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05168a8-2a3d-499b-8e67-786797880c12-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.559683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.559642 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.559775 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.559685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzh8x\" (UniqueName: \"kubernetes.io/projected/d05168a8-2a3d-499b-8e67-786797880c12-kube-api-access-fzh8x\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.660878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.660792 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.660878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.660839 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzh8x\" (UniqueName: \"kubernetes.io/projected/d05168a8-2a3d-499b-8e67-786797880c12-kube-api-access-fzh8x\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.660878 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.660876 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.661122 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.660905 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05168a8-2a3d-499b-8e67-786797880c12-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.661122 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:33:57.660960 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-serving-cert: secret "maas-api-e2e-shared-a-w3-3240ec-serving-cert" not found Sep 19 09:33:57.661122 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:33:57.661018 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls podName:d05168a8-2a3d-499b-8e67-786797880c12 nodeName:}" failed. No retries permitted until 2026-09-19 09:33:58.161003891 +0000 UTC m=+1644.919124899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls") pod "maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" (UID: "d05168a8-2a3d-499b-8e67-786797880c12") : secret "maas-api-e2e-shared-a-w3-3240ec-serving-cert" not found Sep 19 09:33:57.661683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.661658 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05168a8-2a3d-499b-8e67-786797880c12-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.663383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.663362 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:57.669664 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:57.669641 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzh8x\" (UniqueName: \"kubernetes.io/projected/d05168a8-2a3d-499b-8e67-786797880c12-kube-api-access-fzh8x\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:58.165842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:58.165808 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:58.168255 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:58.168236 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:58.293171 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:58.293132 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:58.570439 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:58.570409 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449"] Sep 19 09:33:58.572607 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:33:58.572568 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05168a8_2a3d_499b_8e67_786797880c12.slice/crio-b21695d284dc200ec8c9865dcd77a74f808a64a4f31849abf4d7d6f370dfef60 WatchSource:0}: Error finding container b21695d284dc200ec8c9865dcd77a74f808a64a4f31849abf4d7d6f370dfef60: Status 404 returned error can't find the container with id b21695d284dc200ec8c9865dcd77a74f808a64a4f31849abf4d7d6f370dfef60 Sep 19 09:33:58.830332 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:58.830243 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" event={"ID":"d05168a8-2a3d-499b-8e67-786797880c12","Type":"ContainerStarted","Data":"b21695d284dc200ec8c9865dcd77a74f808a64a4f31849abf4d7d6f370dfef60"} Sep 19 09:33:59.838815 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:59.838779 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" event={"ID":"d05168a8-2a3d-499b-8e67-786797880c12","Type":"ContainerStarted","Data":"acf18641f4a929514b22a636fadb24f23cba32559317a480185e9c9d384e9ca6"} Sep 19 09:33:59.839209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:59.838929 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:33:59.873668 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:33:59.873619 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" podStartSLOduration=2.325887326 podStartE2EDuration="2.873602327s" totalImagesPullingTime="547.715001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:33:59.871450006 +0000 UTC m=+1646.629571031" watchObservedRunningTime="2026-09-19 09:33:59.873602327 +0000 UTC m=+1646.631723379" Sep 19 09:34:05.850832 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:05.850798 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:34:14.929382 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:14.929349 2585 generic.go:356] "Generic (PLEG): container finished" podID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerID="53a5bbcf2c219166a6896b9e8c2fdfc7cf030e680212fb8c1c1b175b784e3dc2" exitCode=0 Sep 19 09:34:14.929789 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:14.929388 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" event={"ID":"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc","Type":"ContainerDied","Data":"53a5bbcf2c219166a6896b9e8c2fdfc7cf030e680212fb8c1c1b175b784e3dc2"} Sep 19 09:34:14.929789 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:14.929782 2585 scope.go:117] "RemoveContainer" containerID="53a5bbcf2c219166a6896b9e8c2fdfc7cf030e680212fb8c1c1b175b784e3dc2" Sep 19 09:34:14.930936 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:14.930900 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:14.931048 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:14.930971 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:14.931048 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:14.931002 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:15.941230 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:15.941184 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" event={"ID":"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc","Type":"ContainerStarted","Data":"7d7e490a08bf396242ff2aa4de50490f64536d23c594f1e186b7009125b9275c"} Sep 19 09:34:16.341672 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:16.341638 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:34:16.341858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:16.341827 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 19 09:34:16.341912 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:16.341894 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 19 09:34:17.342036 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:17.342001 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 19 09:34:17.342408 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:17.342074 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 19 09:34:18.341769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:18.341735 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 19 09:34:18.341938 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:18.341784 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 19 09:34:19.342443 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:19.342408 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 19 09:34:19.342853 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:19.342466 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 19 09:34:20.341932 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:20.341892 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Sep 19 09:34:20.342127 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:20.341950 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Sep 19 09:34:21.429294 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:21.429254 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:34:21.438773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:21.438748 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:34:21.438920 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:21.438844 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:34:24.410670 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.410634 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk"] Sep 19 09:34:24.411307 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.411289 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" Sep 19 09:34:24.411307 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.411307 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d36ec3b2-5604-468e-8cec-f5b4160cfa09" containerName="istio-proxy" Sep 19 09:34:24.416946 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.416916 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.419667 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.419645 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-1afa13-openshift-default-dockercfg-v7qsq\"" Sep 19 09:34:24.427929 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.427895 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk"] Sep 19 09:34:24.528154 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528124 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57bt8\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-kube-api-access-57bt8\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528303 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528162 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-data\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528303 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528182 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-token\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528303 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528223 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-certs\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528303 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-envoy\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528463 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528325 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-socket\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528463 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528342 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-credential-socket\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528463 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528382 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c5d2a3-1b5a-4684-a590-e02b08511345-istiod-ca-cert\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.528463 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.528452 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-podinfo\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629581 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629542 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57bt8\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-kube-api-access-57bt8\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629593 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-data\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629613 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-token\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629767 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629630 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-certs\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629764 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-envoy\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629820 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-socket\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629849 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-credential-socket\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.629947 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629926 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c5d2a3-1b5a-4684-a590-e02b08511345-istiod-ca-cert\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.630142 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.629968 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-podinfo\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.630142 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.630033 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-certs\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.630142 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.630075 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-data\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.630261 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.630158 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-socket\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.630299 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.630257 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-credential-socket\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.630612 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.630591 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c5d2a3-1b5a-4684-a590-e02b08511345-istiod-ca-cert\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.632085 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.632053 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-envoy\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.632415 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.632399 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-podinfo\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.640422 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.640398 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-token\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.640568 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.640491 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57bt8\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-kube-api-access-57bt8\") pod \"e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:24.732215 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:24.732142 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:25.010843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:25.010814 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk"] Sep 19 09:34:25.013447 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:34:25.013422 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c5d2a3_1b5a_4684_a590_e02b08511345.slice/crio-b346c90dd762b553bc1711acd9c5ed24b6d0078cf9d1d275199757c1b1465850 WatchSource:0}: Error finding container b346c90dd762b553bc1711acd9c5ed24b6d0078cf9d1d275199757c1b1465850: Status 404 returned error can't find the container with id b346c90dd762b553bc1711acd9c5ed24b6d0078cf9d1d275199757c1b1465850 Sep 19 09:34:25.016377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:25.016345 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:25.016457 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:25.016417 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:25.016457 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:25.016447 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:26.001531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:26.001477 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" event={"ID":"d8c5d2a3-1b5a-4684-a590-e02b08511345","Type":"ContainerStarted","Data":"d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b"} Sep 19 09:34:26.002015 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:26.001603 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" event={"ID":"d8c5d2a3-1b5a-4684-a590-e02b08511345","Type":"ContainerStarted","Data":"b346c90dd762b553bc1711acd9c5ed24b6d0078cf9d1d275199757c1b1465850"} Sep 19 09:34:26.024331 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:26.024283 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" podStartSLOduration=2.024268835 podStartE2EDuration="2.024268835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:34:26.023631615 +0000 UTC m=+1672.781752641" watchObservedRunningTime="2026-09-19 09:34:26.024268835 +0000 UTC m=+1672.782389862" Sep 19 09:34:26.732927 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:26.732873 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:26.738254 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:26.738224 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:27.008430 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:27.008336 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:27.009471 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:27.009451 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:32.273318 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.273283 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg"] Sep 19 09:34:32.277202 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.277185 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.279833 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.279806 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-1afa13-serving-cert\"" Sep 19 09:34:32.287342 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.287320 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg"] Sep 19 09:34:32.407633 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.407596 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32475e30-e2f3-4b91-8844-69d36182bd74-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.407633 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.407635 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87fbc\" (UniqueName: \"kubernetes.io/projected/32475e30-e2f3-4b91-8844-69d36182bd74-kube-api-access-87fbc\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.407874 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.407664 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.407874 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.407715 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.508597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.508556 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32475e30-e2f3-4b91-8844-69d36182bd74-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.508812 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.508608 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87fbc\" (UniqueName: \"kubernetes.io/projected/32475e30-e2f3-4b91-8844-69d36182bd74-kube-api-access-87fbc\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.508812 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.508651 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.508812 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.508686 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.508989 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:34:32.508841 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-serving-cert: secret "maas-api-e2e-shared-b-w3-1afa13-serving-cert" not found Sep 19 09:34:32.508989 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:34:32.508918 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls podName:32475e30-e2f3-4b91-8844-69d36182bd74 nodeName:}" failed. No retries permitted until 2026-09-19 09:34:33.008895872 +0000 UTC m=+1679.767016894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls") pod "maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" (UID: "32475e30-e2f3-4b91-8844-69d36182bd74") : secret "maas-api-e2e-shared-b-w3-1afa13-serving-cert" not found Sep 19 09:34:32.509367 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.509340 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32475e30-e2f3-4b91-8844-69d36182bd74-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.511351 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.511322 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:32.517787 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:32.517744 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87fbc\" (UniqueName: \"kubernetes.io/projected/32475e30-e2f3-4b91-8844-69d36182bd74-kube-api-access-87fbc\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:33.014770 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:33.014724 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:33.018023 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:33.017991 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:33.189003 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:33.188964 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:33.480074 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:33.480045 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg"] Sep 19 09:34:33.482941 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:34:33.482906 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32475e30_e2f3_4b91_8844_69d36182bd74.slice/crio-555699bacb98845302ce08c7dd0236ecdf3480a10c70de226fc493f4a65d135e WatchSource:0}: Error finding container 555699bacb98845302ce08c7dd0236ecdf3480a10c70de226fc493f4a65d135e: Status 404 returned error can't find the container with id 555699bacb98845302ce08c7dd0236ecdf3480a10c70de226fc493f4a65d135e Sep 19 09:34:33.484357 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:33.484341 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:34:34.062217 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:34.062174 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" event={"ID":"32475e30-e2f3-4b91-8844-69d36182bd74","Type":"ContainerStarted","Data":"fa54ca987ab1869490cadd90f6fd7812e00aaebc9f2d718491b266016dae095b"} Sep 19 09:34:34.062408 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:34.062234 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" event={"ID":"32475e30-e2f3-4b91-8844-69d36182bd74","Type":"ContainerStarted","Data":"555699bacb98845302ce08c7dd0236ecdf3480a10c70de226fc493f4a65d135e"} Sep 19 09:34:34.062408 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:34.062261 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:34.083164 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:34.083114 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" podStartSLOduration=1.768196123 podStartE2EDuration="2.083094688s" totalImagesPullingTime="314.898565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:34:34.08268442 +0000 UTC m=+1680.840805446" watchObservedRunningTime="2026-09-19 09:34:34.083094688 +0000 UTC m=+1680.841215715" Sep 19 09:34:40.073389 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:40.073361 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:34:55.196280 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:55.196240 2585 generic.go:356] "Generic (PLEG): container finished" podID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerID="d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b" exitCode=0 Sep 19 09:34:55.196709 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:55.196308 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" event={"ID":"d8c5d2a3-1b5a-4684-a590-e02b08511345","Type":"ContainerDied","Data":"d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b"} Sep 19 09:34:55.196785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:55.196721 2585 scope.go:117] "RemoveContainer" containerID="d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b" Sep 19 09:34:55.197841 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:55.197816 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:55.197914 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:55.197884 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:55.197951 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:55.197916 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:34:56.204437 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:56.204400 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" event={"ID":"d8c5d2a3-1b5a-4684-a590-e02b08511345","Type":"ContainerStarted","Data":"eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba"} Sep 19 09:34:56.735544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:56.734673 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:56.735544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:56.734843 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Sep 19 09:34:56.735544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:56.734905 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Sep 19 09:34:57.008658 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:57.008580 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:34:57.732959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:57.732906 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Sep 19 09:34:57.733438 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:57.733021 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Sep 19 09:34:58.732751 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:58.732716 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Sep 19 09:34:58.733047 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:58.733018 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Sep 19 09:34:59.733230 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:59.733193 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Sep 19 09:34:59.733831 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:34:59.733797 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Sep 19 09:35:00.733278 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:00.733231 2585 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Sep 19 09:35:00.733786 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:00.733318 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Sep 19 09:35:01.736374 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:01.736347 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:35:01.737255 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:01.737233 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:35:40.833260 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:40.833227 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt"] Sep 19 09:35:40.833832 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:40.833585 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" podUID="f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" containerName="maas-api" containerID="cri-o://38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23" gracePeriod=30 Sep 19 09:35:41.110368 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.110344 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:35:41.188790 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.188756 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvzvk\" (UniqueName: \"kubernetes.io/projected/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-kube-api-access-fvzvk\") pod \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " Sep 19 09:35:41.188964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.188808 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-metrics-certs\") pod \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " Sep 19 09:35:41.188964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.188857 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-openshift-service-ca\") pod \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " Sep 19 09:35:41.188964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.188881 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls\") pod \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\" (UID: \"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a\") " Sep 19 09:35:41.189221 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.189193 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" (UID: "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:35:41.189361 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.189340 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:35:41.190950 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.190920 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-kube-api-access-fvzvk" (OuterVolumeSpecName: "kube-api-access-fvzvk") pod "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" (UID: "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a"). InnerVolumeSpecName "kube-api-access-fvzvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:35:41.191047 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.190999 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" (UID: "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:35:41.191097 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.191063 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" (UID: "f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:35:41.290541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.290496 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvzvk\" (UniqueName: \"kubernetes.io/projected/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-kube-api-access-fvzvk\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:35:41.290541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.290546 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:35:41.290733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.290556 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:35:41.505343 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.505315 2585 generic.go:356] "Generic (PLEG): container finished" podID="f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" containerID="38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23" exitCode=0 Sep 19 09:35:41.505483 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.505356 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" Sep 19 09:35:41.505483 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.505356 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" event={"ID":"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a","Type":"ContainerDied","Data":"38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23"} Sep 19 09:35:41.505483 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.505461 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt" event={"ID":"f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a","Type":"ContainerDied","Data":"ae05efbdc67087edfd9beb703fcd319033566cc3f995aa72bad8e4f880c31cfe"} Sep 19 09:35:41.505658 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.505483 2585 scope.go:117] "RemoveContainer" containerID="38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23" Sep 19 09:35:41.515340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.515320 2585 scope.go:117] "RemoveContainer" containerID="38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23" Sep 19 09:35:41.515668 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:35:41.515641 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23\": container with ID starting with 38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23 not found: ID does not exist" containerID="38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23" Sep 19 09:35:41.515754 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.515674 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23"} err="failed to get container status \"38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23\": rpc error: code = NotFound desc = could not find container \"38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23\": container with ID starting with 38ef990a0fa1ef33a8f8d27440c7308e81873cb02d8df76977663cdd37e23e23 not found: ID does not exist" Sep 19 09:35:41.525305 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.525280 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt"] Sep 19 09:35:41.533866 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:41.533846 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e46994f4-b94f6785b-wsnvt"] Sep 19 09:35:43.437284 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:35:43.437251 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" path="/var/lib/kubelet/pods/f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a/volumes" Sep 19 09:36:35.569052 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:36:35.569022 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:36:35.572264 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:36:35.572244 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:36:35.578462 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:36:35.578443 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:36:35.581933 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:36:35.581916 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:37:26.939793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:26.939752 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9"] Sep 19 09:37:26.940459 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:26.940436 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" containerName="maas-api" Sep 19 09:37:26.940552 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:26.940463 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" containerName="maas-api" Sep 19 09:37:26.940614 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:26.940596 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f294dee3-40a0-4d3c-aa8b-eba68c1f0c2a" containerName="maas-api" Sep 19 09:37:26.945574 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:26.945474 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:26.952050 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:26.948900 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-9e9462-openshift-default-dockercfg-zltvj\"" Sep 19 09:37:26.957372 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:26.957344 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9"] Sep 19 09:37:27.016271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016210 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-credential-socket\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016268 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-certs\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016344 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-token\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016407 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-envoy\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016464 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-data\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016531 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkh5s\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-kube-api-access-mkh5s\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016587 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-socket\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016641 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istiod-ca-cert\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.016858 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.016682 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-podinfo\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.117635 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117600 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-podinfo\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.117842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117663 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-credential-socket\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.117842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117792 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-certs\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.117974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117842 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-token\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.117974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117885 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-envoy\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.117974 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117937 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-data\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118125 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117976 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-credential-socket\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118125 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.117979 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkh5s\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-kube-api-access-mkh5s\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118125 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.118026 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-socket\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118125 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.118080 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istiod-ca-cert\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118530 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.118494 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-socket\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118632 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.118545 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-certs\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.118666 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-data\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.118814 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.118796 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istiod-ca-cert\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.120535 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.120499 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-podinfo\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.120735 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.120716 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-envoy\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.127176 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.127156 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-token\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.127550 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.127506 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkh5s\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-kube-api-access-mkh5s\") pod \"e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.260477 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.260388 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:27.538565 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.538443 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9"] Sep 19 09:37:27.541006 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:37:27.540975 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92620489_ec9c_4b7f_a9f4_e3594ab43a5c.slice/crio-afa4ced07a44c7c72f3c0fdc02c41b5e9bbc7dd37d54856c8a5840ae57a2d2c4 WatchSource:0}: Error finding container afa4ced07a44c7c72f3c0fdc02c41b5e9bbc7dd37d54856c8a5840ae57a2d2c4: Status 404 returned error can't find the container with id afa4ced07a44c7c72f3c0fdc02c41b5e9bbc7dd37d54856c8a5840ae57a2d2c4 Sep 19 09:37:27.543269 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.543237 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:37:27.543348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.543303 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:37:27.543348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:27.543328 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:37:28.137773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:28.137734 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" event={"ID":"92620489-ec9c-4b7f-a9f4-e3594ab43a5c","Type":"ContainerStarted","Data":"1bfef45355b93916a68cefb5de55841a536297c880c3524e880c21fd43ceaf96"} Sep 19 09:37:28.138257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:28.137797 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" event={"ID":"92620489-ec9c-4b7f-a9f4-e3594ab43a5c","Type":"ContainerStarted","Data":"afa4ced07a44c7c72f3c0fdc02c41b5e9bbc7dd37d54856c8a5840ae57a2d2c4"} Sep 19 09:37:28.162978 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:28.162930 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podStartSLOduration=2.162913905 podStartE2EDuration="2.162913905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:37:28.16145854 +0000 UTC m=+1854.919579566" watchObservedRunningTime="2026-09-19 09:37:28.162913905 +0000 UTC m=+1854.921034929" Sep 19 09:37:28.260707 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:28.260671 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:28.266001 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:28.265975 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:29.144257 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:29.144227 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:29.145255 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:29.145236 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:37:34.190093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.190059 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z"] Sep 19 09:37:34.194222 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.194200 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.196903 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.196881 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-9e9462-serving-cert\"" Sep 19 09:37:34.203461 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.203381 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z"] Sep 19 09:37:34.290103 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.290066 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141f122-6844-4d71-838d-3beaf39c3b2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.290312 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.290172 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn5pl\" (UniqueName: \"kubernetes.io/projected/5141f122-6844-4d71-838d-3beaf39c3b2c-kube-api-access-fn5pl\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.290491 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.290457 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-metrics-certs\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.290699 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.290656 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-maas-api-tls\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.391426 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.391392 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-metrics-certs\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.391641 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.391437 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-maas-api-tls\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.391641 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.391486 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141f122-6844-4d71-838d-3beaf39c3b2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.391641 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.391513 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn5pl\" (UniqueName: \"kubernetes.io/projected/5141f122-6844-4d71-838d-3beaf39c3b2c-kube-api-access-fn5pl\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.392230 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.392208 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141f122-6844-4d71-838d-3beaf39c3b2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.393859 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.393841 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-maas-api-tls\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.393970 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.393952 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-metrics-certs\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.399663 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.399637 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn5pl\" (UniqueName: \"kubernetes.io/projected/5141f122-6844-4d71-838d-3beaf39c3b2c-kube-api-access-fn5pl\") pod \"maas-api-e2e-api-a-9e9462-869989b449-hdz9z\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.509419 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.509336 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:34.795167 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:34.795131 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z"] Sep 19 09:37:34.795980 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:37:34.795956 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5141f122_6844_4d71_838d_3beaf39c3b2c.slice/crio-8d54625fb2c2845acd7e746450b51ba03c9f2be916d2405db7d1218af722c7cc WatchSource:0}: Error finding container 8d54625fb2c2845acd7e746450b51ba03c9f2be916d2405db7d1218af722c7cc: Status 404 returned error can't find the container with id 8d54625fb2c2845acd7e746450b51ba03c9f2be916d2405db7d1218af722c7cc Sep 19 09:37:35.181450 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:35.181412 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" event={"ID":"5141f122-6844-4d71-838d-3beaf39c3b2c","Type":"ContainerStarted","Data":"8d54625fb2c2845acd7e746450b51ba03c9f2be916d2405db7d1218af722c7cc"} Sep 19 09:37:36.189088 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:36.189046 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" event={"ID":"5141f122-6844-4d71-838d-3beaf39c3b2c","Type":"ContainerStarted","Data":"e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9"} Sep 19 09:37:36.189478 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:36.189263 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:37:36.209431 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:36.209390 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" podStartSLOduration=1.759235264 podStartE2EDuration="2.209376665s" totalImagesPullingTime="450.141401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:37:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:37:36.206935625 +0000 UTC m=+1862.965056650" watchObservedRunningTime="2026-09-19 09:37:36.209376665 +0000 UTC m=+1862.967497689" Sep 19 09:37:42.199687 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:37:42.199658 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:38:00.146154 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.146109 2585 patch_prober.go:28] interesting pod/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Sep 19 09:38:00.148528 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.146171 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Sep 19 09:38:00.337628 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.337589 2585 generic.go:356] "Generic (PLEG): container finished" podID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerID="1bfef45355b93916a68cefb5de55841a536297c880c3524e880c21fd43ceaf96" exitCode=0 Sep 19 09:38:00.337799 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.337667 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" event={"ID":"92620489-ec9c-4b7f-a9f4-e3594ab43a5c","Type":"ContainerDied","Data":"1bfef45355b93916a68cefb5de55841a536297c880c3524e880c21fd43ceaf96"} Sep 19 09:38:00.338108 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.338095 2585 scope.go:117] "RemoveContainer" containerID="1bfef45355b93916a68cefb5de55841a536297c880c3524e880c21fd43ceaf96" Sep 19 09:38:00.339245 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.339219 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:00.339312 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.339288 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:00.339348 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.339316 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:00.847815 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.847781 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb"] Sep 19 09:38:00.851589 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.851564 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.854115 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.854096 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0304ec-openshift-default-dockercfg-7ld5r\"" Sep 19 09:38:00.870239 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.870210 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb"] Sep 19 09:38:00.962830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962505 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-envoy\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.962830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962636 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-credential-socket\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.962830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962708 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-token\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.962830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962738 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-socket\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.962830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962769 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-podinfo\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.962830 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962800 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-certs\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.963367 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962892 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac3e205-a2ef-4874-ac64-e22113437c73-istiod-ca-cert\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.963367 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962951 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-data\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:00.963367 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:00.962979 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h8h4\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-kube-api-access-7h8h4\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064399 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064365 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-podinfo\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064399 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064408 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-certs\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064455 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac3e205-a2ef-4874-ac64-e22113437c73-istiod-ca-cert\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064485 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-data\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064502 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h8h4\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-kube-api-access-7h8h4\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064585 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-envoy\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064627 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-credential-socket\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064906 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064731 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-token\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.064906 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.064783 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-socket\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.065028 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.065001 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-data\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.065112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.065088 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-certs\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.065225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.065204 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-socket\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.065335 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.065316 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-credential-socket\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.065377 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.065329 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac3e205-a2ef-4874-ac64-e22113437c73-istiod-ca-cert\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.066971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.066948 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-envoy\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.067209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.067189 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-podinfo\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.086834 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.086807 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-token\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.089445 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.089412 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h8h4\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-kube-api-access-7h8h4\") pod \"e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.162250 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.162148 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:01.346135 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.346095 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" event={"ID":"92620489-ec9c-4b7f-a9f4-e3594ab43a5c","Type":"ContainerStarted","Data":"a257c6befd02628f1d1550e1211bda518469bf6411ad7c33170e7f9aab4fa84c"} Sep 19 09:38:01.472323 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.472289 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb"] Sep 19 09:38:01.472638 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:38:01.472615 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac3e205_a2ef_4874_ac64_e22113437c73.slice/crio-cf0dc975730a8f9b0ff4c69f895b72ec2bc5e2c5bd1383b6cbed79925dad95ba WatchSource:0}: Error finding container cf0dc975730a8f9b0ff4c69f895b72ec2bc5e2c5bd1383b6cbed79925dad95ba: Status 404 returned error can't find the container with id cf0dc975730a8f9b0ff4c69f895b72ec2bc5e2c5bd1383b6cbed79925dad95ba Sep 19 09:38:01.475093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.475063 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:01.475192 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.475127 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:01.475192 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:01.475155 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:02.261218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:02.261181 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:38:02.261642 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:02.261387 2585 patch_prober.go:28] interesting pod/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Sep 19 09:38:02.261642 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:02.261434 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Sep 19 09:38:02.357597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:02.357556 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" event={"ID":"6ac3e205-a2ef-4874-ac64-e22113437c73","Type":"ContainerStarted","Data":"9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b"} Sep 19 09:38:02.357789 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:02.357621 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" event={"ID":"6ac3e205-a2ef-4874-ac64-e22113437c73","Type":"ContainerStarted","Data":"cf0dc975730a8f9b0ff4c69f895b72ec2bc5e2c5bd1383b6cbed79925dad95ba"} Sep 19 09:38:02.381987 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:02.381933 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podStartSLOduration=2.381915938 podStartE2EDuration="2.381915938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:38:02.381132663 +0000 UTC m=+1889.139253689" watchObservedRunningTime="2026-09-19 09:38:02.381915938 +0000 UTC m=+1889.140036962" Sep 19 09:38:03.162818 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:03.162775 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:03.168275 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:03.168243 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:03.261326 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:03.261296 2585 patch_prober.go:28] interesting pod/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Sep 19 09:38:03.261728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:03.261388 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Sep 19 09:38:03.364795 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:03.364765 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:03.365639 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:03.365620 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:04.260964 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:04.260920 2585 patch_prober.go:28] interesting pod/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Sep 19 09:38:04.261153 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:04.260984 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Sep 19 09:38:05.261801 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:05.261761 2585 patch_prober.go:28] interesting pod/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Sep 19 09:38:05.262232 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:05.261852 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Sep 19 09:38:06.261837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:06.261787 2585 patch_prober.go:28] interesting pod/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Sep 19 09:38:06.262288 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:06.261869 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Sep 19 09:38:07.265709 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:07.265673 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:38:07.266181 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:07.266121 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:38:07.267036 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:07.267011 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:38:27.825058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:27.818967 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx"] Sep 19 09:38:27.825058 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:27.819188 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" podUID="e429751d-4a55-4aa3-a9f9-0c62c669045c" containerName="maas-api" containerID="cri-o://2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8" gracePeriod=30 Sep 19 09:38:28.171822 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.171794 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:38:28.265888 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.265847 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dx6v\" (UniqueName: \"kubernetes.io/projected/e429751d-4a55-4aa3-a9f9-0c62c669045c-kube-api-access-2dx6v\") pod \"e429751d-4a55-4aa3-a9f9-0c62c669045c\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " Sep 19 09:38:28.266049 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.265902 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls\") pod \"e429751d-4a55-4aa3-a9f9-0c62c669045c\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " Sep 19 09:38:28.266049 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.266045 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-metrics-certs\") pod \"e429751d-4a55-4aa3-a9f9-0c62c669045c\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " Sep 19 09:38:28.266185 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.266130 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e429751d-4a55-4aa3-a9f9-0c62c669045c-openshift-service-ca\") pod \"e429751d-4a55-4aa3-a9f9-0c62c669045c\" (UID: \"e429751d-4a55-4aa3-a9f9-0c62c669045c\") " Sep 19 09:38:28.266748 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.266705 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e429751d-4a55-4aa3-a9f9-0c62c669045c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e429751d-4a55-4aa3-a9f9-0c62c669045c" (UID: "e429751d-4a55-4aa3-a9f9-0c62c669045c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:38:28.268836 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.268511 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e429751d-4a55-4aa3-a9f9-0c62c669045c" (UID: "e429751d-4a55-4aa3-a9f9-0c62c669045c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:38:28.268836 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.268595 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e429751d-4a55-4aa3-a9f9-0c62c669045c" (UID: "e429751d-4a55-4aa3-a9f9-0c62c669045c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:38:28.269139 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.269098 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e429751d-4a55-4aa3-a9f9-0c62c669045c-kube-api-access-2dx6v" (OuterVolumeSpecName: "kube-api-access-2dx6v") pod "e429751d-4a55-4aa3-a9f9-0c62c669045c" (UID: "e429751d-4a55-4aa3-a9f9-0c62c669045c"). InnerVolumeSpecName "kube-api-access-2dx6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:38:28.367860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.367824 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e429751d-4a55-4aa3-a9f9-0c62c669045c-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:28.367860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.367852 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dx6v\" (UniqueName: \"kubernetes.io/projected/e429751d-4a55-4aa3-a9f9-0c62c669045c-kube-api-access-2dx6v\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:28.367860 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.367866 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:28.368091 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.367879 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e429751d-4a55-4aa3-a9f9-0c62c669045c-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:28.532653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.532563 2585 generic.go:356] "Generic (PLEG): container finished" podID="e429751d-4a55-4aa3-a9f9-0c62c669045c" containerID="2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8" exitCode=0 Sep 19 09:38:28.532653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.532617 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" event={"ID":"e429751d-4a55-4aa3-a9f9-0c62c669045c","Type":"ContainerDied","Data":"2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8"} Sep 19 09:38:28.532883 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.532665 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" event={"ID":"e429751d-4a55-4aa3-a9f9-0c62c669045c","Type":"ContainerDied","Data":"bc87cffa46f22cac422366578f04fad679cc0b8e3d660c3a30186cbb167fae01"} Sep 19 09:38:28.532883 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.532693 2585 scope.go:117] "RemoveContainer" containerID="2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8" Sep 19 09:38:28.532883 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.532622 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx" Sep 19 09:38:28.543333 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.543311 2585 scope.go:117] "RemoveContainer" containerID="2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8" Sep 19 09:38:28.543725 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:38:28.543701 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8\": container with ID starting with 2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8 not found: ID does not exist" containerID="2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8" Sep 19 09:38:28.543801 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.543734 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8"} err="failed to get container status \"2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8\": rpc error: code = NotFound desc = could not find container \"2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8\": container with ID starting with 2f348a14fac5b1b154aac0d1af7b3cc41e090f0d7c4af686c97351fffb5a6be8 not found: ID does not exist" Sep 19 09:38:28.560985 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.560961 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx"] Sep 19 09:38:28.565256 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:28.565233 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-14e3f6-5ccd5774bc-dgcrx"] Sep 19 09:38:29.437920 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:29.437887 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e429751d-4a55-4aa3-a9f9-0c62c669045c" path="/var/lib/kubelet/pods/e429751d-4a55-4aa3-a9f9-0c62c669045c/volumes" Sep 19 09:38:31.554731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:31.554634 2585 generic.go:356] "Generic (PLEG): container finished" podID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerID="9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b" exitCode=0 Sep 19 09:38:31.554731 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:31.554692 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" event={"ID":"6ac3e205-a2ef-4874-ac64-e22113437c73","Type":"ContainerDied","Data":"9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b"} Sep 19 09:38:31.555155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:31.555128 2585 scope.go:117] "RemoveContainer" containerID="9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b" Sep 19 09:38:31.556561 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:31.556512 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:31.556677 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:31.556613 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:31.556677 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:31.556663 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:38:32.564541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:32.564486 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" event={"ID":"6ac3e205-a2ef-4874-ac64-e22113437c73","Type":"ContainerStarted","Data":"d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081"} Sep 19 09:38:33.162912 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:33.162879 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:33.164287 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:33.164257 2585 patch_prober.go:28] interesting pod/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Sep 19 09:38:33.164436 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:33.164334 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Sep 19 09:38:33.365318 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:33.365282 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:34.163536 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:34.163488 2585 patch_prober.go:28] interesting pod/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Sep 19 09:38:34.163950 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:34.163564 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Sep 19 09:38:35.163008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:35.162972 2585 patch_prober.go:28] interesting pod/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Sep 19 09:38:35.163189 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:35.163029 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Sep 19 09:38:36.163043 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:36.163005 2585 patch_prober.go:28] interesting pod/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Sep 19 09:38:36.163451 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:36.163065 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Sep 19 09:38:37.163117 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:37.163075 2585 patch_prober.go:28] interesting pod/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Sep 19 09:38:37.163630 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:37.163136 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Sep 19 09:38:38.163238 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:38.163200 2585 patch_prober.go:28] interesting pod/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Sep 19 09:38:38.163726 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:38.163260 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Sep 19 09:38:39.166811 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:39.166776 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:39.167857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:39.167833 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:38:52.670569 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:52.670530 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z"] Sep 19 09:38:52.671075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:52.670767 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" podUID="5141f122-6844-4d71-838d-3beaf39c3b2c" containerName="maas-api" containerID="cri-o://e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9" gracePeriod=30 Sep 19 09:38:52.936247 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:52.936186 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:38:53.036531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.036488 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn5pl\" (UniqueName: \"kubernetes.io/projected/5141f122-6844-4d71-838d-3beaf39c3b2c-kube-api-access-fn5pl\") pod \"5141f122-6844-4d71-838d-3beaf39c3b2c\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " Sep 19 09:38:53.036685 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.036589 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-maas-api-tls\") pod \"5141f122-6844-4d71-838d-3beaf39c3b2c\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " Sep 19 09:38:53.036685 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.036630 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141f122-6844-4d71-838d-3beaf39c3b2c-openshift-service-ca\") pod \"5141f122-6844-4d71-838d-3beaf39c3b2c\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " Sep 19 09:38:53.036798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.036774 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-metrics-certs\") pod \"5141f122-6844-4d71-838d-3beaf39c3b2c\" (UID: \"5141f122-6844-4d71-838d-3beaf39c3b2c\") " Sep 19 09:38:53.037094 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.037063 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5141f122-6844-4d71-838d-3beaf39c3b2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5141f122-6844-4d71-838d-3beaf39c3b2c" (UID: "5141f122-6844-4d71-838d-3beaf39c3b2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:38:53.037222 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.037168 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5141f122-6844-4d71-838d-3beaf39c3b2c-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:53.038963 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.038936 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5141f122-6844-4d71-838d-3beaf39c3b2c-kube-api-access-fn5pl" (OuterVolumeSpecName: "kube-api-access-fn5pl") pod "5141f122-6844-4d71-838d-3beaf39c3b2c" (UID: "5141f122-6844-4d71-838d-3beaf39c3b2c"). InnerVolumeSpecName "kube-api-access-fn5pl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:38:53.039176 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.039148 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5141f122-6844-4d71-838d-3beaf39c3b2c" (UID: "5141f122-6844-4d71-838d-3beaf39c3b2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:38:53.039256 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.039193 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5141f122-6844-4d71-838d-3beaf39c3b2c" (UID: "5141f122-6844-4d71-838d-3beaf39c3b2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:38:53.138168 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.138126 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:53.138168 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.138169 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5141f122-6844-4d71-838d-3beaf39c3b2c-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:53.138376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.138185 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn5pl\" (UniqueName: \"kubernetes.io/projected/5141f122-6844-4d71-838d-3beaf39c3b2c-kube-api-access-fn5pl\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:38:53.709446 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.709360 2585 generic.go:356] "Generic (PLEG): container finished" podID="5141f122-6844-4d71-838d-3beaf39c3b2c" containerID="e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9" exitCode=0 Sep 19 09:38:53.709446 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.709417 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" Sep 19 09:38:53.709446 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.709424 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" event={"ID":"5141f122-6844-4d71-838d-3beaf39c3b2c","Type":"ContainerDied","Data":"e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9"} Sep 19 09:38:53.709937 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.709483 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z" event={"ID":"5141f122-6844-4d71-838d-3beaf39c3b2c","Type":"ContainerDied","Data":"8d54625fb2c2845acd7e746450b51ba03c9f2be916d2405db7d1218af722c7cc"} Sep 19 09:38:53.709937 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.709511 2585 scope.go:117] "RemoveContainer" containerID="e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9" Sep 19 09:38:53.720020 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.720002 2585 scope.go:117] "RemoveContainer" containerID="e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9" Sep 19 09:38:53.720325 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:38:53.720307 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9\": container with ID starting with e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9 not found: ID does not exist" containerID="e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9" Sep 19 09:38:53.720373 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.720335 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9"} err="failed to get container status \"e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9\": rpc error: code = NotFound desc = could not find container \"e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9\": container with ID starting with e18ea545f98a84793401f68fe3310757e4e8ce0673c88a4db10f680a0376a6a9 not found: ID does not exist" Sep 19 09:38:53.727453 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.727429 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z"] Sep 19 09:38:53.730369 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:53.730348 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-9e9462-869989b449-hdz9z"] Sep 19 09:38:55.436949 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:55.436907 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5141f122-6844-4d71-838d-3beaf39c3b2c" path="/var/lib/kubelet/pods/5141f122-6844-4d71-838d-3beaf39c3b2c/volumes" Sep 19 09:38:59.656774 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:59.656741 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9"] Sep 19 09:38:59.657236 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:38:59.656944 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" containerID="cri-o://a257c6befd02628f1d1550e1211bda518469bf6411ad7c33170e7f9aab4fa84c" gracePeriod=30 Sep 19 09:39:04.785261 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:04.785227 2585 generic.go:356] "Generic (PLEG): container finished" podID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerID="a257c6befd02628f1d1550e1211bda518469bf6411ad7c33170e7f9aab4fa84c" exitCode=0 Sep 19 09:39:04.785663 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:04.785301 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" event={"ID":"92620489-ec9c-4b7f-a9f4-e3594ab43a5c","Type":"ContainerDied","Data":"a257c6befd02628f1d1550e1211bda518469bf6411ad7c33170e7f9aab4fa84c"} Sep 19 09:39:04.785663 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:04.785377 2585 scope.go:117] "RemoveContainer" containerID="1bfef45355b93916a68cefb5de55841a536297c880c3524e880c21fd43ceaf96" Sep 19 09:39:04.921013 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:04.920948 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:39:05.052081 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052054 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-envoy\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052113 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-podinfo\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052140 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-socket\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052161 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-certs\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052208 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istiod-ca-cert\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052303 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-credential-socket\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052336 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkh5s\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-kube-api-access-mkh5s\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052368 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-token\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052396 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-data\") pod \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\" (UID: \"92620489-ec9c-4b7f-a9f4-e3594ab43a5c\") " Sep 19 09:39:05.052494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052467 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:05.052798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052607 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:05.052798 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052679 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:39:05.052902 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052817 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:05.052902 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052837 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:05.052902 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052881 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.052902 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052901 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.053108 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.052916 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.054458 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.054422 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:39:05.054458 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.054449 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:05.054689 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.054484 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-kube-api-access-mkh5s" (OuterVolumeSpecName: "kube-api-access-mkh5s") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "kube-api-access-mkh5s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:39:05.054689 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.054579 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "92620489-ec9c-4b7f-a9f4-e3594ab43a5c" (UID: "92620489-ec9c-4b7f-a9f4-e3594ab43a5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:39:05.154052 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.154018 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkh5s\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-kube-api-access-mkh5s\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.154052 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.154045 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.154052 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.154054 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.154052 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.154062 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.154318 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.154070 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.154318 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.154077 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92620489-ec9c-4b7f-a9f4-e3594ab43a5c-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:05.792774 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.792686 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" Sep 19 09:39:05.793186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.792690 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9" event={"ID":"92620489-ec9c-4b7f-a9f4-e3594ab43a5c","Type":"ContainerDied","Data":"afa4ced07a44c7c72f3c0fdc02c41b5e9bbc7dd37d54856c8a5840ae57a2d2c4"} Sep 19 09:39:05.793186 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.792822 2585 scope.go:117] "RemoveContainer" containerID="a257c6befd02628f1d1550e1211bda518469bf6411ad7c33170e7f9aab4fa84c" Sep 19 09:39:05.812581 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.812553 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9"] Sep 19 09:39:05.816587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:05.816563 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-9e9462-openshift-default-576b6849cd-gvhf9"] Sep 19 09:39:07.437054 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:07.437018 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" path="/var/lib/kubelet/pods/92620489-ec9c-4b7f-a9f4-e3594ab43a5c/volumes" Sep 19 09:39:22.353076 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:22.353045 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb"] Sep 19 09:39:22.353506 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:22.353276 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" containerID="cri-o://d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081" gracePeriod=30 Sep 19 09:39:24.169347 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:24.169309 2585 patch_prober.go:28] interesting pod/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 19 09:39:24.169725 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:24.169401 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 09:39:27.642156 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.642127 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:39:27.678946 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.678918 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-token\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.678966 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-socket\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679001 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac3e205-a2ef-4874-ac64-e22113437c73-istiod-ca-cert\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679042 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-certs\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679091 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-credential-socket\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679329 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679126 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-podinfo\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679329 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679197 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-data\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679329 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679251 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-envoy\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679329 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679288 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h8h4\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-kube-api-access-7h8h4\") pod \"6ac3e205-a2ef-4874-ac64-e22113437c73\" (UID: \"6ac3e205-a2ef-4874-ac64-e22113437c73\") " Sep 19 09:39:27.679329 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679312 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:27.679636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679349 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:27.679636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679403 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:27.679636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679469 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:27.679636 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679543 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac3e205-a2ef-4874-ac64-e22113437c73-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:39:27.679837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679742 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.679837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679763 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.679837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679778 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ac3e205-a2ef-4874-ac64-e22113437c73-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.679837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679792 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.679837 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.679804 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.681539 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.681505 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:39:27.681709 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.681688 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:27.681795 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.681730 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:39:27.681870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.681850 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-kube-api-access-7h8h4" (OuterVolumeSpecName: "kube-api-access-7h8h4") pod "6ac3e205-a2ef-4874-ac64-e22113437c73" (UID: "6ac3e205-a2ef-4874-ac64-e22113437c73"). InnerVolumeSpecName "kube-api-access-7h8h4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:39:27.780503 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.780469 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.780503 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.780499 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h8h4\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-kube-api-access-7h8h4\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.780717 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.780512 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.780717 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.780554 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ac3e205-a2ef-4874-ac64-e22113437c73-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:27.941469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.941391 2585 generic.go:356] "Generic (PLEG): container finished" podID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerID="d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081" exitCode=0 Sep 19 09:39:27.941469 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.941447 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" Sep 19 09:39:27.941722 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.941493 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" event={"ID":"6ac3e205-a2ef-4874-ac64-e22113437c73","Type":"ContainerDied","Data":"d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081"} Sep 19 09:39:27.941722 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.941568 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb" event={"ID":"6ac3e205-a2ef-4874-ac64-e22113437c73","Type":"ContainerDied","Data":"cf0dc975730a8f9b0ff4c69f895b72ec2bc5e2c5bd1383b6cbed79925dad95ba"} Sep 19 09:39:27.941722 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.941601 2585 scope.go:117] "RemoveContainer" containerID="d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081" Sep 19 09:39:27.951600 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.951581 2585 scope.go:117] "RemoveContainer" containerID="9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b" Sep 19 09:39:27.959773 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.959757 2585 scope.go:117] "RemoveContainer" containerID="d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081" Sep 19 09:39:27.960043 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:39:27.960023 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081\": container with ID starting with d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081 not found: ID does not exist" containerID="d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081" Sep 19 09:39:27.960112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.960051 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081"} err="failed to get container status \"d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081\": rpc error: code = NotFound desc = could not find container \"d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081\": container with ID starting with d604c1b832b202f31d49c1fcc6219f4e41513de9f3b02ffb40b409e259745081 not found: ID does not exist" Sep 19 09:39:27.960112 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.960077 2585 scope.go:117] "RemoveContainer" containerID="9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b" Sep 19 09:39:27.960327 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:39:27.960311 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b\": container with ID starting with 9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b not found: ID does not exist" containerID="9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b" Sep 19 09:39:27.960378 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.960335 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b"} err="failed to get container status \"9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b\": rpc error: code = NotFound desc = could not find container \"9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b\": container with ID starting with 9ffd05d041b0912c359f635bd031d86e0537fba9e6d3bcaccc41a946b6b33b6b not found: ID does not exist" Sep 19 09:39:27.966483 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.966461 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb"] Sep 19 09:39:27.971017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:27.970998 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0304ec-openshift-default-55b47c59fb-z2mmb"] Sep 19 09:39:29.436600 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:29.436569 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" path="/var/lib/kubelet/pods/6ac3e205-a2ef-4874-ac64-e22113437c73/volumes" Sep 19 09:39:40.637363 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:40.637319 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc"] Sep 19 09:39:40.642542 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:40.637901 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" containerID="cri-o://0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b" gracePeriod=30 Sep 19 09:39:45.901032 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:45.900965 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:39:46.048751 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.048714 2585 generic.go:356] "Generic (PLEG): container finished" podID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerID="0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b" exitCode=0 Sep 19 09:39:46.048956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.048774 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" Sep 19 09:39:46.048956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.048790 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" event={"ID":"1ef36802-3461-41a6-93f5-d8d79d0ccc4d","Type":"ContainerDied","Data":"0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b"} Sep 19 09:39:46.048956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.048853 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc" event={"ID":"1ef36802-3461-41a6-93f5-d8d79d0ccc4d","Type":"ContainerDied","Data":"958ed8cdfc1cdd6018b4dedb872ad9e65f13ee24bfb5440692bd51478ec36029"} Sep 19 09:39:46.048956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.048878 2585 scope.go:117] "RemoveContainer" containerID="0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b" Sep 19 09:39:46.058020 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.057997 2585 scope.go:117] "RemoveContainer" containerID="bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353" Sep 19 09:39:46.061709 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.061680 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-envoy\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.061857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.061728 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpl7f\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-kube-api-access-rpl7f\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.061857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.061753 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-token\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.061857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.061782 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istiod-ca-cert\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.061857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.061829 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-certs\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.062080 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.061913 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-socket\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.062209 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.062179 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:46.062396 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.062357 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:46.062586 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.062558 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:46.062586 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.062572 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:39:46.062748 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.062598 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-credential-socket\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.062748 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.062685 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-podinfo\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.062869 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.062764 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-data\") pod \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\" (UID: \"1ef36802-3461-41a6-93f5-d8d79d0ccc4d\") " Sep 19 09:39:46.063163 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.063119 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:46.063163 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.063138 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.063163 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.063157 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.063384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.063172 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.063384 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.063185 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.064210 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.064184 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-kube-api-access-rpl7f" (OuterVolumeSpecName: "kube-api-access-rpl7f") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "kube-api-access-rpl7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:39:46.064300 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.064281 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:39:46.064545 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.064491 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:39:46.064939 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.064917 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ef36802-3461-41a6-93f5-d8d79d0ccc4d" (UID: "1ef36802-3461-41a6-93f5-d8d79d0ccc4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:39:46.068003 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.067979 2585 scope.go:117] "RemoveContainer" containerID="0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b" Sep 19 09:39:46.068350 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:39:46.068331 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b\": container with ID starting with 0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b not found: ID does not exist" containerID="0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b" Sep 19 09:39:46.068432 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.068360 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b"} err="failed to get container status \"0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b\": rpc error: code = NotFound desc = could not find container \"0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b\": container with ID starting with 0bb160db6351ffb75fd3a3952d453f5743add3b55b07c3f83e8b5c0e7a20a60b not found: ID does not exist" Sep 19 09:39:46.068432 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.068378 2585 scope.go:117] "RemoveContainer" containerID="bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353" Sep 19 09:39:46.068692 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:39:46.068667 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353\": container with ID starting with bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353 not found: ID does not exist" containerID="bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353" Sep 19 09:39:46.068759 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.068701 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353"} err="failed to get container status \"bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353\": rpc error: code = NotFound desc = could not find container \"bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353\": container with ID starting with bfb60d4b5ce95fa5f06cf13c72527ec0fa9998d4936a3b81827c54500e0c4353 not found: ID does not exist" Sep 19 09:39:46.164749 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.164663 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.164749 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.164695 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.164749 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.164703 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.164749 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.164711 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpl7f\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-kube-api-access-rpl7f\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.164749 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.164721 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef36802-3461-41a6-93f5-d8d79d0ccc4d-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:39:46.378670 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.378630 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc"] Sep 19 09:39:46.386354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:46.386324 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-261ce4-openshift-default-74d5b968-gdlgc"] Sep 19 09:39:47.444442 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:47.444405 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" path="/var/lib/kubelet/pods/1ef36802-3461-41a6-93f5-d8d79d0ccc4d/volumes" Sep 19 09:39:58.326944 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:58.326904 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6"] Sep 19 09:39:58.327411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:39:58.327240 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" containerID="cri-o://dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa" gracePeriod=30 Sep 19 09:40:03.600704 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.600637 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:40:03.642462 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642420 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-envoy\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.642682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642511 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-data\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.642682 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642624 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4krz\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-kube-api-access-r4krz\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.642868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642694 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-socket\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.642868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642741 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istiod-ca-cert\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.642868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642775 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-credential-socket\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.642868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642789 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-data" (OuterVolumeSpecName: "istio-data") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:40:03.642868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642828 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-certs\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.642868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642861 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-podinfo\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.643161 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.642894 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-token\") pod \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\" (UID: \"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614\") " Sep 19 09:40:03.643161 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.643126 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:40:03.643292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.643271 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.643349 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.643300 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.643413 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.643391 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:40:03.643651 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.643628 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:40:03.645624 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.645583 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:40:03.645824 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.645783 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:40:03.646540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.646493 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:40:03.646540 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.646497 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-kube-api-access-r4krz" (OuterVolumeSpecName: "kube-api-access-r4krz") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "kube-api-access-r4krz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:40:03.646666 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.646574 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-token" (OuterVolumeSpecName: "istio-token") pod "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" (UID: "2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:40:03.744292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.744257 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4krz\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-kube-api-access-r4krz\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.744292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.744285 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.744292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.744294 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.744292 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.744301 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.744593 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.744310 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.744593 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.744317 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:03.744593 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:03.744324 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:40:04.170111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.170075 2585 generic.go:356] "Generic (PLEG): container finished" podID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerID="dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa" exitCode=0 Sep 19 09:40:04.170333 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.170130 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" Sep 19 09:40:04.170333 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.170169 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" event={"ID":"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614","Type":"ContainerDied","Data":"dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa"} Sep 19 09:40:04.170333 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.170234 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6" event={"ID":"2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614","Type":"ContainerDied","Data":"74b8b86e4940fa8f87fa6df6f7c09637dd05789c954e51e42c4a36fdca58b7e1"} Sep 19 09:40:04.170333 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.170258 2585 scope.go:117] "RemoveContainer" containerID="dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa" Sep 19 09:40:04.180003 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.179984 2585 scope.go:117] "RemoveContainer" containerID="2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182" Sep 19 09:40:04.187893 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.187872 2585 scope.go:117] "RemoveContainer" containerID="dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa" Sep 19 09:40:04.188150 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:40:04.188134 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa\": container with ID starting with dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa not found: ID does not exist" containerID="dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa" Sep 19 09:40:04.188206 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.188156 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa"} err="failed to get container status \"dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa\": rpc error: code = NotFound desc = could not find container \"dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa\": container with ID starting with dc3742ee592178c9e92bcbc73fb7700f836c486c0cdbc69e2cd8aa58766424aa not found: ID does not exist" Sep 19 09:40:04.188206 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.188171 2585 scope.go:117] "RemoveContainer" containerID="2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182" Sep 19 09:40:04.188390 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:40:04.188373 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182\": container with ID starting with 2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182 not found: ID does not exist" containerID="2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182" Sep 19 09:40:04.188431 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.188394 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182"} err="failed to get container status \"2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182\": rpc error: code = NotFound desc = could not find container \"2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182\": container with ID starting with 2eec6a62f9bb3f3bad0a393931b28372386e97e204d01930af70b7e4c4eab182 not found: ID does not exist" Sep 19 09:40:04.196485 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.196463 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6"] Sep 19 09:40:04.209439 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:04.209420 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-66c0fb-openshift-default-6c95b49dbb-t4bx6"] Sep 19 09:40:05.437111 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:40:05.437071 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" path="/var/lib/kubelet/pods/2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614/volumes" Sep 19 09:41:35.609716 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:41:35.609682 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:41:35.612997 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:41:35.612973 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:41:35.624332 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:41:35.624313 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:41:35.627477 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:41:35.627462 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:43:06.194240 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.194199 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq"] Sep 19 09:43:06.194976 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.194944 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" Sep 19 09:43:06.194976 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.194970 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.194987 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.194995 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195010 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195018 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195038 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195046 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195054 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195063 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195105 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5141f122-6844-4d71-838d-3beaf39c3b2c" containerName="maas-api" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195113 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5141f122-6844-4d71-838d-3beaf39c3b2c" containerName="maas-api" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195130 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e429751d-4a55-4aa3-a9f9-0c62c669045c" containerName="maas-api" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195143 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e429751d-4a55-4aa3-a9f9-0c62c669045c" containerName="maas-api" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195158 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" Sep 19 09:43:06.195223 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195165 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" Sep 19 09:43:06.195998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195298 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" Sep 19 09:43:06.195998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195312 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" Sep 19 09:43:06.195998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195321 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5141f122-6844-4d71-838d-3beaf39c3b2c" containerName="maas-api" Sep 19 09:43:06.195998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195328 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" Sep 19 09:43:06.195998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195339 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" Sep 19 09:43:06.195998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195348 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e429751d-4a55-4aa3-a9f9-0c62c669045c" containerName="maas-api" Sep 19 09:43:06.195998 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.195356 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ac3e205-a2ef-4874-ac64-e22113437c73" containerName="istio-proxy" Sep 19 09:43:06.198804 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.198783 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.201281 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.201261 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e10409-serving-cert\"" Sep 19 09:43:06.214457 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.214424 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq"] Sep 19 09:43:06.316973 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.316927 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.317165 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.316991 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-metrics-certs\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.317165 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.317048 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/79cef0a6-f890-4d11-afa5-da685fc8d2ff-kube-api-access-j7dpg\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.317165 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.317105 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cef0a6-f890-4d11-afa5-da685fc8d2ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.417882 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.417840 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/79cef0a6-f890-4d11-afa5-da685fc8d2ff-kube-api-access-j7dpg\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.418086 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.417920 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cef0a6-f890-4d11-afa5-da685fc8d2ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.418086 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.417978 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.418086 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.418022 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-metrics-certs\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.418262 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:43:06.418156 2585 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-serving-cert: secret "maas-api-e2e-ipp-b-e10409-serving-cert" not found Sep 19 09:43:06.418262 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:43:06.418252 2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls podName:79cef0a6-f890-4d11-afa5-da685fc8d2ff nodeName:}" failed. No retries permitted until 2026-09-19 09:43:06.918227712 +0000 UTC m=+2193.676348732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls") pod "maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" (UID: "79cef0a6-f890-4d11-afa5-da685fc8d2ff") : secret "maas-api-e2e-ipp-b-e10409-serving-cert" not found Sep 19 09:43:06.418577 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.418556 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cef0a6-f890-4d11-afa5-da685fc8d2ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.420582 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.420556 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-metrics-certs\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.427297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.427263 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/79cef0a6-f890-4d11-afa5-da685fc8d2ff-kube-api-access-j7dpg\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.924324 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.924281 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:06.926801 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:06.926775 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:07.109402 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:07.109369 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:07.399436 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:07.399407 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq"] Sep 19 09:43:07.401557 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:43:07.401508 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79cef0a6_f890_4d11_afa5_da685fc8d2ff.slice/crio-50a95511467d4a77b46300ec44d02fdc6ccc241bc0e27875738943569444255e WatchSource:0}: Error finding container 50a95511467d4a77b46300ec44d02fdc6ccc241bc0e27875738943569444255e: Status 404 returned error can't find the container with id 50a95511467d4a77b46300ec44d02fdc6ccc241bc0e27875738943569444255e Sep 19 09:43:07.402955 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:07.402937 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:43:08.228607 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:08.228556 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" event={"ID":"79cef0a6-f890-4d11-afa5-da685fc8d2ff","Type":"ContainerStarted","Data":"6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea"} Sep 19 09:43:08.228796 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:08.228637 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:43:08.228796 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:08.228661 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" event={"ID":"79cef0a6-f890-4d11-afa5-da685fc8d2ff","Type":"ContainerStarted","Data":"50a95511467d4a77b46300ec44d02fdc6ccc241bc0e27875738943569444255e"} Sep 19 09:43:08.248879 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:08.248832 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" podStartSLOduration=1.924840667 podStartE2EDuration="2.24881698s" totalImagesPullingTime="323.976313ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:43:08.247920568 +0000 UTC m=+2195.006041593" watchObservedRunningTime="2026-09-19 09:43:08.24881698 +0000 UTC m=+2195.006938005" Sep 19 09:43:14.239816 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:43:14.239785 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:44:11.679738 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.679658 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n"] Sep 19 09:44:11.682212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.680058 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" Sep 19 09:44:11.682212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.680069 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" Sep 19 09:44:11.682212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.680083 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" Sep 19 09:44:11.682212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.680088 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" Sep 19 09:44:11.682212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.680150 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef36802-3461-41a6-93f5-d8d79d0ccc4d" containerName="istio-proxy" Sep 19 09:44:11.682212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.680157 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="92620489-ec9c-4b7f-a9f4-e3594ab43a5c" containerName="istio-proxy" Sep 19 09:44:11.682212 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.680164 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab42e91-b4b2-4753-ad4f-dcbe2c9b5614" containerName="istio-proxy" Sep 19 09:44:11.683067 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.683052 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.685899 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.685875 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-9f4a5b-openshift-default-dockercfg-vhv6m\"" Sep 19 09:44:11.703591 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.703565 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n"] Sep 19 09:44:11.739945 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.739913 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-socket\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.739954 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqh4g\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-kube-api-access-cqh4g\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.740019 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-podinfo\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.740064 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-data\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740238 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.740091 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-token\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740238 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.740180 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-certs\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.740234 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-credential-socket\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.740277 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/908b0062-d7b4-424c-a70e-ab24750d0ad3-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.740451 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.740347 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-envoy\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.841905 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.841873 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-envoy\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.841926 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-socket\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842039 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqh4g\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-kube-api-access-cqh4g\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842084 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-podinfo\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842228 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842124 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-data\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842228 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842155 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-token\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842228 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-certs\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842251 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-socket\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842272 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-credential-socket\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842330 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842311 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/908b0062-d7b4-424c-a70e-ab24750d0ad3-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.842960 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.842931 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-certs\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.843090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.843067 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-data\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.843339 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.843159 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-credential-socket\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.844090 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.844069 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/908b0062-d7b4-424c-a70e-ab24750d0ad3-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.846850 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.846823 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-envoy\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.852349 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.852327 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-token\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.852754 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.852729 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqh4g\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-kube-api-access-cqh4g\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.855597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.855575 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-podinfo\") pod \"e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:11.993496 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:11.993402 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:12.278029 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.277999 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n"] Sep 19 09:44:12.279493 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:44:12.279466 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod908b0062_d7b4_424c_a70e_ab24750d0ad3.slice/crio-94dea75ac28688367f253530bf7e642a285d14958f9169af5d962a9617b84509 WatchSource:0}: Error finding container 94dea75ac28688367f253530bf7e642a285d14958f9169af5d962a9617b84509: Status 404 returned error can't find the container with id 94dea75ac28688367f253530bf7e642a285d14958f9169af5d962a9617b84509 Sep 19 09:44:12.284872 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.284839 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:44:12.285000 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.284913 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:44:12.285000 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.284942 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:44:12.595109 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.595076 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" event={"ID":"908b0062-d7b4-424c-a70e-ab24750d0ad3","Type":"ContainerStarted","Data":"b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934"} Sep 19 09:44:12.595307 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.595128 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" event={"ID":"908b0062-d7b4-424c-a70e-ab24750d0ad3","Type":"ContainerStarted","Data":"94dea75ac28688367f253530bf7e642a285d14958f9169af5d962a9617b84509"} Sep 19 09:44:12.621380 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.621315 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podStartSLOduration=1.621294053 podStartE2EDuration="1.621294053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:44:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:44:12.619011511 +0000 UTC m=+2259.377132536" watchObservedRunningTime="2026-09-19 09:44:12.621294053 +0000 UTC m=+2259.379415077" Sep 19 09:44:12.994495 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:12.994410 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:13.999786 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:13.999753 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:14.606380 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:14.606346 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:14.607354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:14.607331 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:34.725911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:34.725817 2585 generic.go:356] "Generic (PLEG): container finished" podID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerID="b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934" exitCode=0 Sep 19 09:44:34.725911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:34.725891 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" event={"ID":"908b0062-d7b4-424c-a70e-ab24750d0ad3","Type":"ContainerDied","Data":"b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934"} Sep 19 09:44:34.726354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:34.726336 2585 scope.go:117] "RemoveContainer" containerID="b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934" Sep 19 09:44:34.727391 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:34.727363 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:44:34.727478 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:34.727428 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:44:34.727478 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:34.727459 2585 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"} Sep 19 09:44:35.733552 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:35.733505 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" event={"ID":"908b0062-d7b4-424c-a70e-ab24750d0ad3","Type":"ContainerStarted","Data":"731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2"} Sep 19 09:44:35.993750 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:35.993663 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:35.993900 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:35.993874 2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Sep 19 09:44:35.993957 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:35.993931 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Sep 19 09:44:36.994141 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:36.994099 2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Sep 19 09:44:36.994643 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:36.994176 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Sep 19 09:44:37.994017 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:37.993981 2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Sep 19 09:44:37.994197 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:37.994064 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Sep 19 09:44:38.994597 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:38.994565 2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Sep 19 09:44:38.995015 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:38.994636 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Sep 19 09:44:39.993861 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:39.993826 2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Sep 19 09:44:39.994051 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:39.993881 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Sep 19 09:44:40.994023 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:40.993989 2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Sep 19 09:44:40.994422 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:40.994041 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Sep 19 09:44:41.994225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:41.994193 2585 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body= Sep 19 09:44:41.994630 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:41.994251 2585 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" Sep 19 09:44:42.997968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:42.997937 2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:42.998345 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:42.998202 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:44:42.998918 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:44:42.998898 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:45:03.634084 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:03.634052 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n"] Sep 19 09:45:03.634610 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:03.634249 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" containerID="cri-o://731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2" gracePeriod=30 Sep 19 09:45:08.883374 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.883342 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:45:08.923451 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.923357 2585 generic.go:356] "Generic (PLEG): container finished" podID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerID="731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2" exitCode=0 Sep 19 09:45:08.923451 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.923417 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" Sep 19 09:45:08.923451 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.923438 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" event={"ID":"908b0062-d7b4-424c-a70e-ab24750d0ad3","Type":"ContainerDied","Data":"731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2"} Sep 19 09:45:08.923785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.923493 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n" event={"ID":"908b0062-d7b4-424c-a70e-ab24750d0ad3","Type":"ContainerDied","Data":"94dea75ac28688367f253530bf7e642a285d14958f9169af5d962a9617b84509"} Sep 19 09:45:08.923785 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.923514 2585 scope.go:117] "RemoveContainer" containerID="731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2" Sep 19 09:45:08.935907 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.935887 2585 scope.go:117] "RemoveContainer" containerID="b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934" Sep 19 09:45:08.946683 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.946657 2585 scope.go:117] "RemoveContainer" containerID="731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2" Sep 19 09:45:08.947233 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:45:08.947194 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2\": container with ID starting with 731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2 not found: ID does not exist" containerID="731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2" Sep 19 09:45:08.947408 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.947377 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2"} err="failed to get container status \"731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2\": rpc error: code = NotFound desc = could not find container \"731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2\": container with ID starting with 731ecc031ae41221aec9dc539807a93cfaa9bca21213cf67b2f2d1ec07d0cce2 not found: ID does not exist" Sep 19 09:45:08.947665 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.947646 2585 scope.go:117] "RemoveContainer" containerID="b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934" Sep 19 09:45:08.948227 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:45:08.948196 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934\": container with ID starting with b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934 not found: ID does not exist" containerID="b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934" Sep 19 09:45:08.948338 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:08.948240 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934"} err="failed to get container status \"b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934\": rpc error: code = NotFound desc = could not find container \"b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934\": container with ID starting with b0ca6009e9837b1a308667613240ed3c0ad244b510d69942ae372451d63a0934 not found: ID does not exist" Sep 19 09:45:09.015626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015589 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-data\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.015792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015648 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-certs\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.015792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015689 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/908b0062-d7b4-424c-a70e-ab24750d0ad3-istiod-ca-cert\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.015792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015747 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-credential-socket\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.015792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015771 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-socket\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.016025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015803 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-token\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.016025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015829 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-envoy\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.016025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015897 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqh4g\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-kube-api-access-cqh4g\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.016025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.015938 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-data" (OuterVolumeSpecName: "istio-data") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:45:09.016025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016013 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-podinfo\") pod \"908b0062-d7b4-424c-a70e-ab24750d0ad3\" (UID: \"908b0062-d7b4-424c-a70e-ab24750d0ad3\") " Sep 19 09:45:09.016345 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016114 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:45:09.016345 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016271 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:45:09.016461 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016443 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.016538 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016466 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.016538 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016479 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.016726 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016691 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:45:09.016726 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.016706 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/908b0062-d7b4-424c-a70e-ab24750d0ad3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:45:09.018436 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.018409 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-kube-api-access-cqh4g" (OuterVolumeSpecName: "kube-api-access-cqh4g") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "kube-api-access-cqh4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:45:09.018589 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.018553 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:45:09.018944 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.018920 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-token" (OuterVolumeSpecName: "istio-token") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:45:09.019051 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.018972 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "908b0062-d7b4-424c-a70e-ab24750d0ad3" (UID: "908b0062-d7b4-424c-a70e-ab24750d0ad3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:45:09.117258 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.117226 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqh4g\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-kube-api-access-cqh4g\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.117258 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.117255 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.117258 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.117266 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.117489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.117277 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/908b0062-d7b4-424c-a70e-ab24750d0ad3-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.117489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.117284 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.117489 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.117291 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/908b0062-d7b4-424c-a70e-ab24750d0ad3-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:45:09.247453 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.247425 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n"] Sep 19 09:45:09.253970 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.253944 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9f4a5b-openshift-default-6897ff6f46-9sn9n"] Sep 19 09:45:09.437622 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:45:09.437589 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" path="/var/lib/kubelet/pods/908b0062-d7b4-424c-a70e-ab24750d0ad3/volumes" Sep 19 09:46:35.650842 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:35.650814 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:46:35.654198 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:35.654179 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:46:35.663037 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:35.663011 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:46:35.666042 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:35.666023 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:46:37.598465 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.598428 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq"] Sep 19 09:46:37.599048 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.598725 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" podUID="79cef0a6-f890-4d11-afa5-da685fc8d2ff" containerName="maas-api" containerID="cri-o://6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea" gracePeriod=30 Sep 19 09:46:37.855721 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.855640 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:46:37.941437 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.941397 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-metrics-certs\") pod \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " Sep 19 09:46:37.941643 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.941496 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/79cef0a6-f890-4d11-afa5-da685fc8d2ff-kube-api-access-j7dpg\") pod \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " Sep 19 09:46:37.941643 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.941586 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cef0a6-f890-4d11-afa5-da685fc8d2ff-openshift-service-ca\") pod \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " Sep 19 09:46:37.941643 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.941620 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls\") pod \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\" (UID: \"79cef0a6-f890-4d11-afa5-da685fc8d2ff\") " Sep 19 09:46:37.942005 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.941978 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79cef0a6-f890-4d11-afa5-da685fc8d2ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79cef0a6-f890-4d11-afa5-da685fc8d2ff" (UID: "79cef0a6-f890-4d11-afa5-da685fc8d2ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:46:37.943900 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.943871 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79cef0a6-f890-4d11-afa5-da685fc8d2ff-kube-api-access-j7dpg" (OuterVolumeSpecName: "kube-api-access-j7dpg") pod "79cef0a6-f890-4d11-afa5-da685fc8d2ff" (UID: "79cef0a6-f890-4d11-afa5-da685fc8d2ff"). InnerVolumeSpecName "kube-api-access-j7dpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:46:37.944049 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.944030 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79cef0a6-f890-4d11-afa5-da685fc8d2ff" (UID: "79cef0a6-f890-4d11-afa5-da685fc8d2ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:46:37.944155 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:37.944139 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "79cef0a6-f890-4d11-afa5-da685fc8d2ff" (UID: "79cef0a6-f890-4d11-afa5-da685fc8d2ff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:46:38.042856 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.042818 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79cef0a6-f890-4d11-afa5-da685fc8d2ff-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:38.042856 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.042852 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:38.042856 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.042861 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79cef0a6-f890-4d11-afa5-da685fc8d2ff-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:38.043093 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.042868 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7dpg\" (UniqueName: \"kubernetes.io/projected/79cef0a6-f890-4d11-afa5-da685fc8d2ff-kube-api-access-j7dpg\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:38.431817 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.431780 2585 generic.go:356] "Generic (PLEG): container finished" podID="79cef0a6-f890-4d11-afa5-da685fc8d2ff" containerID="6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea" exitCode=0 Sep 19 09:46:38.432008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.431830 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" Sep 19 09:46:38.432008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.431874 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" event={"ID":"79cef0a6-f890-4d11-afa5-da685fc8d2ff","Type":"ContainerDied","Data":"6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea"} Sep 19 09:46:38.432008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.431924 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq" event={"ID":"79cef0a6-f890-4d11-afa5-da685fc8d2ff","Type":"ContainerDied","Data":"50a95511467d4a77b46300ec44d02fdc6ccc241bc0e27875738943569444255e"} Sep 19 09:46:38.432008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.431949 2585 scope.go:117] "RemoveContainer" containerID="6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea" Sep 19 09:46:38.441912 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.441897 2585 scope.go:117] "RemoveContainer" containerID="6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea" Sep 19 09:46:38.442207 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:46:38.442188 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea\": container with ID starting with 6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea not found: ID does not exist" containerID="6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea" Sep 19 09:46:38.442265 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.442215 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea"} err="failed to get container status \"6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea\": rpc error: code = NotFound desc = could not find container \"6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea\": container with ID starting with 6d68a011633fb3609a91f9727b238b0a7ccbb104a849efe66df0d2246d13a2ea not found: ID does not exist" Sep 19 09:46:38.454242 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.454214 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq"] Sep 19 09:46:38.458776 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:38.458755 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e10409-57746b8b45-chkgq"] Sep 19 09:46:39.437553 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:39.437503 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79cef0a6-f890-4d11-afa5-da685fc8d2ff" path="/var/lib/kubelet/pods/79cef0a6-f890-4d11-afa5-da685fc8d2ff/volumes" Sep 19 09:46:55.275352 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.274445 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449"] Sep 19 09:46:55.275352 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.275284 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" podUID="d05168a8-2a3d-499b-8e67-786797880c12" containerName="maas-api" containerID="cri-o://acf18641f4a929514b22a636fadb24f23cba32559317a480185e9c9d384e9ca6" gracePeriod=30 Sep 19 09:46:55.533180 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.533117 2585 generic.go:356] "Generic (PLEG): container finished" podID="d05168a8-2a3d-499b-8e67-786797880c12" containerID="acf18641f4a929514b22a636fadb24f23cba32559317a480185e9c9d384e9ca6" exitCode=0 Sep 19 09:46:55.533291 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.533185 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" event={"ID":"d05168a8-2a3d-499b-8e67-786797880c12","Type":"ContainerDied","Data":"acf18641f4a929514b22a636fadb24f23cba32559317a480185e9c9d384e9ca6"} Sep 19 09:46:55.533291 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.533236 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" event={"ID":"d05168a8-2a3d-499b-8e67-786797880c12","Type":"ContainerDied","Data":"b21695d284dc200ec8c9865dcd77a74f808a64a4f31849abf4d7d6f370dfef60"} Sep 19 09:46:55.533291 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.533251 2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b21695d284dc200ec8c9865dcd77a74f808a64a4f31849abf4d7d6f370dfef60" Sep 19 09:46:55.534686 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.534670 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:46:55.612477 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.612439 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls\") pod \"d05168a8-2a3d-499b-8e67-786797880c12\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " Sep 19 09:46:55.612680 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.612500 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05168a8-2a3d-499b-8e67-786797880c12-openshift-service-ca\") pod \"d05168a8-2a3d-499b-8e67-786797880c12\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " Sep 19 09:46:55.612680 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.612544 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzh8x\" (UniqueName: \"kubernetes.io/projected/d05168a8-2a3d-499b-8e67-786797880c12-kube-api-access-fzh8x\") pod \"d05168a8-2a3d-499b-8e67-786797880c12\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " Sep 19 09:46:55.612680 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.612651 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-metrics-certs\") pod \"d05168a8-2a3d-499b-8e67-786797880c12\" (UID: \"d05168a8-2a3d-499b-8e67-786797880c12\") " Sep 19 09:46:55.612959 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.612911 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d05168a8-2a3d-499b-8e67-786797880c12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d05168a8-2a3d-499b-8e67-786797880c12" (UID: "d05168a8-2a3d-499b-8e67-786797880c12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:46:55.614971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.614946 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d05168a8-2a3d-499b-8e67-786797880c12" (UID: "d05168a8-2a3d-499b-8e67-786797880c12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:46:55.615177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.615152 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d05168a8-2a3d-499b-8e67-786797880c12-kube-api-access-fzh8x" (OuterVolumeSpecName: "kube-api-access-fzh8x") pod "d05168a8-2a3d-499b-8e67-786797880c12" (UID: "d05168a8-2a3d-499b-8e67-786797880c12"). InnerVolumeSpecName "kube-api-access-fzh8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:46:55.615345 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.615321 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d05168a8-2a3d-499b-8e67-786797880c12" (UID: "d05168a8-2a3d-499b-8e67-786797880c12"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:46:55.713541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.713494 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:55.713701 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.713569 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d05168a8-2a3d-499b-8e67-786797880c12-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:55.713701 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.713590 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzh8x\" (UniqueName: \"kubernetes.io/projected/d05168a8-2a3d-499b-8e67-786797880c12-kube-api-access-fzh8x\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:55.713701 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:55.713605 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d05168a8-2a3d-499b-8e67-786797880c12-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:46:56.539751 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:56.539720 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449" Sep 19 09:46:56.569151 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:56.569123 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449"] Sep 19 09:46:56.574902 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:56.574876 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3240ec-5b8b98b6d9-h9449"] Sep 19 09:46:57.436539 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:46:57.436489 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d05168a8-2a3d-499b-8e67-786797880c12" path="/var/lib/kubelet/pods/d05168a8-2a3d-499b-8e67-786797880c12/volumes" Sep 19 09:47:02.473530 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:02.473483 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b"] Sep 19 09:47:02.473910 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:02.473739 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" containerID="cri-o://7d7e490a08bf396242ff2aa4de50490f64536d23c594f1e186b7009125b9275c" gracePeriod=30 Sep 19 09:47:06.432217 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:06.432183 2585 patch_prober.go:28] interesting pod/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 19 09:47:06.432620 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:06.432268 2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 19 09:47:07.603359 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.603321 2585 generic.go:356] "Generic (PLEG): container finished" podID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerID="7d7e490a08bf396242ff2aa4de50490f64536d23c594f1e186b7009125b9275c" exitCode=0 Sep 19 09:47:07.603764 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.603400 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" event={"ID":"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc","Type":"ContainerDied","Data":"7d7e490a08bf396242ff2aa4de50490f64536d23c594f1e186b7009125b9275c"} Sep 19 09:47:07.603764 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.603470 2585 scope.go:117] "RemoveContainer" containerID="53a5bbcf2c219166a6896b9e8c2fdfc7cf030e680212fb8c1c1b175b784e3dc2" Sep 19 09:47:07.730946 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.730921 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:47:07.824246 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824224 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-credential-socket\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824252 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-data\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824318 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istiod-ca-cert\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824358 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxblt\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-kube-api-access-fxblt\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824411 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824382 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-certs\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824417 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-podinfo\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824445 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-socket\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824468 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-envoy\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824539 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-token\") pod \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\" (UID: \"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc\") " Sep 19 09:47:07.824653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824572 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:07.824653 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824637 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824731 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824808 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824813 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824896 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824914 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824928 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824948 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.824962 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.824962 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.826691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.826664 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-kube-api-access-fxblt" (OuterVolumeSpecName: "kube-api-access-fxblt") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "kube-api-access-fxblt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:47:07.826791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.826692 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:47:07.826791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.826693 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:47:07.826791 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.826734 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" (UID: "d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:07.925514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.925482 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxblt\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-kube-api-access-fxblt\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.925514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.925509 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.925514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.925537 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:07.925755 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:07.925545 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:08.610099 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:08.610064 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" Sep 19 09:47:08.610511 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:08.610072 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b" event={"ID":"d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc","Type":"ContainerDied","Data":"5d35be9b11399acbf01ec7a8d5c23dca8f0a56a265cd6e850fd59de4a963dd6b"} Sep 19 09:47:08.610511 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:08.610202 2585 scope.go:117] "RemoveContainer" containerID="7d7e490a08bf396242ff2aa4de50490f64536d23c594f1e186b7009125b9275c" Sep 19 09:47:08.632720 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:08.632695 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b"] Sep 19 09:47:08.637163 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:08.637141 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-3240ec-openshift-default-5456bb4466-kbb2b"] Sep 19 09:47:09.436077 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:09.436048 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" path="/var/lib/kubelet/pods/d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc/volumes" Sep 19 09:47:13.399803 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.399772 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg"] Sep 19 09:47:13.400271 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.400008 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" podUID="32475e30-e2f3-4b91-8844-69d36182bd74" containerName="maas-api" containerID="cri-o://fa54ca987ab1869490cadd90f6fd7812e00aaebc9f2d718491b266016dae095b" gracePeriod=30 Sep 19 09:47:13.641855 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.641825 2585 generic.go:356] "Generic (PLEG): container finished" podID="32475e30-e2f3-4b91-8844-69d36182bd74" containerID="fa54ca987ab1869490cadd90f6fd7812e00aaebc9f2d718491b266016dae095b" exitCode=0 Sep 19 09:47:13.641978 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.641896 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" event={"ID":"32475e30-e2f3-4b91-8844-69d36182bd74","Type":"ContainerDied","Data":"fa54ca987ab1869490cadd90f6fd7812e00aaebc9f2d718491b266016dae095b"} Sep 19 09:47:13.656236 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.656180 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:47:13.783204 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.783169 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32475e30-e2f3-4b91-8844-69d36182bd74-openshift-service-ca\") pod \"32475e30-e2f3-4b91-8844-69d36182bd74\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " Sep 19 09:47:13.783374 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.783243 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls\") pod \"32475e30-e2f3-4b91-8844-69d36182bd74\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " Sep 19 09:47:13.783374 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.783343 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-metrics-certs\") pod \"32475e30-e2f3-4b91-8844-69d36182bd74\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " Sep 19 09:47:13.783471 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.783406 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87fbc\" (UniqueName: \"kubernetes.io/projected/32475e30-e2f3-4b91-8844-69d36182bd74-kube-api-access-87fbc\") pod \"32475e30-e2f3-4b91-8844-69d36182bd74\" (UID: \"32475e30-e2f3-4b91-8844-69d36182bd74\") " Sep 19 09:47:13.783723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.783691 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32475e30-e2f3-4b91-8844-69d36182bd74-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "32475e30-e2f3-4b91-8844-69d36182bd74" (UID: "32475e30-e2f3-4b91-8844-69d36182bd74"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:47:13.785726 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.785700 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "32475e30-e2f3-4b91-8844-69d36182bd74" (UID: "32475e30-e2f3-4b91-8844-69d36182bd74"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:47:13.785933 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.785908 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32475e30-e2f3-4b91-8844-69d36182bd74-kube-api-access-87fbc" (OuterVolumeSpecName: "kube-api-access-87fbc") pod "32475e30-e2f3-4b91-8844-69d36182bd74" (UID: "32475e30-e2f3-4b91-8844-69d36182bd74"). InnerVolumeSpecName "kube-api-access-87fbc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:47:13.786006 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.785979 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "32475e30-e2f3-4b91-8844-69d36182bd74" (UID: "32475e30-e2f3-4b91-8844-69d36182bd74"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:47:13.884203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.884175 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32475e30-e2f3-4b91-8844-69d36182bd74-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:13.884203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.884201 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:13.884203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.884210 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32475e30-e2f3-4b91-8844-69d36182bd74-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:13.884440 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:13.884218 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87fbc\" (UniqueName: \"kubernetes.io/projected/32475e30-e2f3-4b91-8844-69d36182bd74-kube-api-access-87fbc\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:14.649225 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:14.649191 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" Sep 19 09:47:14.649661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:14.649227 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg" event={"ID":"32475e30-e2f3-4b91-8844-69d36182bd74","Type":"ContainerDied","Data":"555699bacb98845302ce08c7dd0236ecdf3480a10c70de226fc493f4a65d135e"} Sep 19 09:47:14.649661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:14.649303 2585 scope.go:117] "RemoveContainer" containerID="fa54ca987ab1869490cadd90f6fd7812e00aaebc9f2d718491b266016dae095b" Sep 19 09:47:14.675562 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:14.675532 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg"] Sep 19 09:47:14.678066 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:14.678037 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-1afa13-778794db8-7vgjg"] Sep 19 09:47:15.436274 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:15.436236 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32475e30-e2f3-4b91-8844-69d36182bd74" path="/var/lib/kubelet/pods/32475e30-e2f3-4b91-8844-69d36182bd74/volumes" Sep 19 09:47:20.640618 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:20.640531 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk"] Sep 19 09:47:20.641037 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:20.640789 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" containerID="cri-o://eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba" gracePeriod=30 Sep 19 09:47:24.663131 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.663088 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-fh89x"] Sep 19 09:47:24.663722 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.663359 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5fbf54c957-fh89x" podUID="efed2e43-a0a7-41c9-a0ff-1def3d7677bb" containerName="manager" containerID="cri-o://96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170" gracePeriod=30 Sep 19 09:47:24.913888 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.913826 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:47:24.992705 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.992677 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-metrics-certs\") pod \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " Sep 19 09:47:24.992863 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.992725 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46hxv\" (UniqueName: \"kubernetes.io/projected/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-kube-api-access-46hxv\") pod \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " Sep 19 09:47:24.992863 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.992771 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-webhook-cert\") pod \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\" (UID: \"efed2e43-a0a7-41c9-a0ff-1def3d7677bb\") " Sep 19 09:47:24.994804 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.994774 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "efed2e43-a0a7-41c9-a0ff-1def3d7677bb" (UID: "efed2e43-a0a7-41c9-a0ff-1def3d7677bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:47:24.994948 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.994889 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "efed2e43-a0a7-41c9-a0ff-1def3d7677bb" (UID: "efed2e43-a0a7-41c9-a0ff-1def3d7677bb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:47:24.995065 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:24.995047 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-kube-api-access-46hxv" (OuterVolumeSpecName: "kube-api-access-46hxv") pod "efed2e43-a0a7-41c9-a0ff-1def3d7677bb" (UID: "efed2e43-a0a7-41c9-a0ff-1def3d7677bb"). InnerVolumeSpecName "kube-api-access-46hxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:47:25.093874 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.093847 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:25.093874 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.093871 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46hxv\" (UniqueName: \"kubernetes.io/projected/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-kube-api-access-46hxv\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:25.093874 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.093881 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/efed2e43-a0a7-41c9-a0ff-1def3d7677bb-webhook-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:25.717370 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.717335 2585 generic.go:356] "Generic (PLEG): container finished" podID="efed2e43-a0a7-41c9-a0ff-1def3d7677bb" containerID="96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170" exitCode=0 Sep 19 09:47:25.717743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.717381 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-fh89x" Sep 19 09:47:25.717743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.717425 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-fh89x" event={"ID":"efed2e43-a0a7-41c9-a0ff-1def3d7677bb","Type":"ContainerDied","Data":"96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170"} Sep 19 09:47:25.717743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.717492 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-fh89x" event={"ID":"efed2e43-a0a7-41c9-a0ff-1def3d7677bb","Type":"ContainerDied","Data":"2fa192126fa52766bac70c5e4a0814b03903276e4fbb457474e750f67d7effc9"} Sep 19 09:47:25.717743 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.717550 2585 scope.go:117] "RemoveContainer" containerID="96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170" Sep 19 09:47:25.740128 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.740101 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-fh89x"] Sep 19 09:47:25.750556 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.750531 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-fh89x"] Sep 19 09:47:25.752718 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.752704 2585 scope.go:117] "RemoveContainer" containerID="96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170" Sep 19 09:47:25.753063 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:47:25.753044 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170\": container with ID starting with 96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170 not found: ID does not exist" containerID="96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170" Sep 19 09:47:25.753135 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.753077 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170"} err="failed to get container status \"96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170\": rpc error: code = NotFound desc = could not find container \"96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170\": container with ID starting with 96a9ae93c0ff99f1932be1c12898b2de873815346e74a4329fd46bab78caa170 not found: ID does not exist" Sep 19 09:47:25.899970 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:25.899949 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:47:26.002383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002294 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57bt8\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-kube-api-access-57bt8\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002342 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-token\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002383 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002383 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-certs\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002698 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002402 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-credential-socket\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002698 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002420 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-socket\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002698 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002448 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c5d2a3-1b5a-4684-a590-e02b08511345-istiod-ca-cert\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002698 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002503 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-data\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002698 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002557 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-envoy\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.002698 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002596 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-podinfo\") pod \"d8c5d2a3-1b5a-4684-a590-e02b08511345\" (UID: \"d8c5d2a3-1b5a-4684-a590-e02b08511345\") " Sep 19 09:47:26.003018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002781 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:26.003018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002798 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:26.003018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002883 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:26.003018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002961 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8c5d2a3-1b5a-4684-a590-e02b08511345-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:47:26.003018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002970 2585 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.003018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.002989 2585 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-credential-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.003018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.003004 2585 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-workload-socket\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.003385 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.003344 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-data" (OuterVolumeSpecName: "istio-data") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:26.004655 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.004634 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-kube-api-access-57bt8" (OuterVolumeSpecName: "kube-api-access-57bt8") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "kube-api-access-57bt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:47:26.005077 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.005050 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-token" (OuterVolumeSpecName: "istio-token") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:47:26.005237 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.005221 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Sep 19 09:47:26.005306 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.005224 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d8c5d2a3-1b5a-4684-a590-e02b08511345" (UID: "d8c5d2a3-1b5a-4684-a590-e02b08511345"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:47:26.103893 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.103861 2585 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8c5d2a3-1b5a-4684-a590-e02b08511345-istiod-ca-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.103893 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.103890 2585 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-data\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.103893 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.103898 2585 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-envoy\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.104118 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.103905 2585 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-podinfo\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.104118 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.103914 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57bt8\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-kube-api-access-57bt8\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.104118 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.103922 2585 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8c5d2a3-1b5a-4684-a590-e02b08511345-istio-token\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:47:26.725177 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.725140 2585 generic.go:356] "Generic (PLEG): container finished" podID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerID="eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba" exitCode=0 Sep 19 09:47:26.725626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.725195 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" Sep 19 09:47:26.725626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.725215 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" event={"ID":"d8c5d2a3-1b5a-4684-a590-e02b08511345","Type":"ContainerDied","Data":"eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba"} Sep 19 09:47:26.725626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.725279 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk" event={"ID":"d8c5d2a3-1b5a-4684-a590-e02b08511345","Type":"ContainerDied","Data":"b346c90dd762b553bc1711acd9c5ed24b6d0078cf9d1d275199757c1b1465850"} Sep 19 09:47:26.725626 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.725303 2585 scope.go:117] "RemoveContainer" containerID="eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba" Sep 19 09:47:26.734928 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.734909 2585 scope.go:117] "RemoveContainer" containerID="d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b" Sep 19 09:47:26.742965 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.742949 2585 scope.go:117] "RemoveContainer" containerID="eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba" Sep 19 09:47:26.743226 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:47:26.743206 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba\": container with ID starting with eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba not found: ID does not exist" containerID="eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba" Sep 19 09:47:26.743272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.743233 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba"} err="failed to get container status \"eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba\": rpc error: code = NotFound desc = could not find container \"eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba\": container with ID starting with eb80bc7bc8061f85b23644ea4332b044795e4711a17f0e9db45f13b413912dba not found: ID does not exist" Sep 19 09:47:26.743272 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.743250 2585 scope.go:117] "RemoveContainer" containerID="d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b" Sep 19 09:47:26.743465 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:47:26.743450 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b\": container with ID starting with d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b not found: ID does not exist" containerID="d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b" Sep 19 09:47:26.743507 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.743471 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b"} err="failed to get container status \"d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b\": rpc error: code = NotFound desc = could not find container \"d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b\": container with ID starting with d20b36150b33c0b0175f06c2a16576865155c845f97ff060bd5458e3e459b86b not found: ID does not exist" Sep 19 09:47:26.753796 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.753775 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk"] Sep 19 09:47:26.755638 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:26.755621 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1afa13-openshift-default-866466f664-mlvjk"] Sep 19 09:47:27.437199 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:27.437165 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" path="/var/lib/kubelet/pods/d8c5d2a3-1b5a-4684-a590-e02b08511345/volumes" Sep 19 09:47:27.437579 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:27.437564 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efed2e43-a0a7-41c9-a0ff-1def3d7677bb" path="/var/lib/kubelet/pods/efed2e43-a0a7-41c9-a0ff-1def3d7677bb/volumes" Sep 19 09:47:35.832630 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:47:35.832601 2585 scope.go:117] "RemoveContainer" containerID="acf18641f4a929514b22a636fadb24f23cba32559317a480185e9c9d384e9ca6" Sep 19 09:48:26.566710 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.566672 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5fbf54c957-c2xqr"] Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567043 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567056 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567069 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567075 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567084 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efed2e43-a0a7-41c9-a0ff-1def3d7677bb" containerName="manager" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567088 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="efed2e43-a0a7-41c9-a0ff-1def3d7677bb" containerName="manager" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567094 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567099 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567107 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567112 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567124 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32475e30-e2f3-4b91-8844-69d36182bd74" containerName="maas-api" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567128 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="32475e30-e2f3-4b91-8844-69d36182bd74" containerName="maas-api" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567135 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79cef0a6-f890-4d11-afa5-da685fc8d2ff" containerName="maas-api" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567139 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="79cef0a6-f890-4d11-afa5-da685fc8d2ff" containerName="maas-api" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567150 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d05168a8-2a3d-499b-8e67-786797880c12" containerName="maas-api" Sep 19 09:48:26.567147 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567155 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d05168a8-2a3d-499b-8e67-786797880c12" containerName="maas-api" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567209 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567216 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="32475e30-e2f3-4b91-8844-69d36182bd74" containerName="maas-api" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567223 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="efed2e43-a0a7-41c9-a0ff-1def3d7677bb" containerName="manager" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567230 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567236 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="79cef0a6-f890-4d11-afa5-da685fc8d2ff" containerName="maas-api" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567242 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d05168a8-2a3d-499b-8e67-786797880c12" containerName="maas-api" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567247 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567252 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567259 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" Sep 19 09:48:26.567661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.567264 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ccb79d-46dc-4ff9-8a99-15d7112ddcdc" containerName="istio-proxy" Sep 19 09:48:26.570309 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.570292 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.574243 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.574214 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 19 09:48:26.574243 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.574228 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9txm5\"" Sep 19 09:48:26.574427 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.574227 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 19 09:48:26.574427 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.574214 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 19 09:48:26.580691 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.580665 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-c2xqr"] Sep 19 09:48:26.695452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.695409 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-metrics-certs\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.695675 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.695481 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-webhook-cert\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.695675 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.695581 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm49m\" (UniqueName: \"kubernetes.io/projected/3b844bd6-1b84-440b-935d-7881b340fa38-kube-api-access-jm49m\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.796587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.796552 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm49m\" (UniqueName: \"kubernetes.io/projected/3b844bd6-1b84-440b-935d-7881b340fa38-kube-api-access-jm49m\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.796744 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.796631 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-metrics-certs\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.796744 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.796693 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-webhook-cert\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.799247 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.799221 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-webhook-cert\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.799389 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.799270 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-metrics-certs\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.805638 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.805610 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm49m\" (UniqueName: \"kubernetes.io/projected/3b844bd6-1b84-440b-935d-7881b340fa38-kube-api-access-jm49m\") pod \"maas-controller-5fbf54c957-c2xqr\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:26.882277 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:26.882160 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:27.154529 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:27.154486 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-c2xqr"] Sep 19 09:48:27.156084 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:48:27.156035 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b844bd6_1b84_440b_935d_7881b340fa38.slice/crio-3519a987a4c80f46facb789eb4545245502a4977e50f62467cd48fbdfc92f395 WatchSource:0}: Error finding container 3519a987a4c80f46facb789eb4545245502a4977e50f62467cd48fbdfc92f395: Status 404 returned error can't find the container with id 3519a987a4c80f46facb789eb4545245502a4977e50f62467cd48fbdfc92f395 Sep 19 09:48:27.157552 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:27.157514 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:48:28.067485 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:28.067446 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" event={"ID":"3b844bd6-1b84-440b-935d-7881b340fa38","Type":"ContainerStarted","Data":"0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541"} Sep 19 09:48:28.067844 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:28.067500 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" event={"ID":"3b844bd6-1b84-440b-935d-7881b340fa38","Type":"ContainerStarted","Data":"3519a987a4c80f46facb789eb4545245502a4977e50f62467cd48fbdfc92f395"} Sep 19 09:48:28.067844 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:28.067648 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:28.084889 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:28.084849 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" podStartSLOduration=1.667606892 podStartE2EDuration="2.084833469s" totalImagesPullingTime="417.226577ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:48:28.084151302 +0000 UTC m=+2514.842272341" watchObservedRunningTime="2026-09-19 09:48:28.084833469 +0000 UTC m=+2514.842954494" Sep 19 09:48:39.075697 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:39.075665 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:48.441236 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.441200 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-c2xqr"] Sep 19 09:48:48.441654 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.441474 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" podUID="3b844bd6-1b84-440b-935d-7881b340fa38" containerName="manager" containerID="cri-o://0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541" gracePeriod=30 Sep 19 09:48:48.686539 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.686491 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:48.788064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.787974 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm49m\" (UniqueName: \"kubernetes.io/projected/3b844bd6-1b84-440b-935d-7881b340fa38-kube-api-access-jm49m\") pod \"3b844bd6-1b84-440b-935d-7881b340fa38\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " Sep 19 09:48:48.788064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.788052 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-metrics-certs\") pod \"3b844bd6-1b84-440b-935d-7881b340fa38\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " Sep 19 09:48:48.788297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.788139 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-webhook-cert\") pod \"3b844bd6-1b84-440b-935d-7881b340fa38\" (UID: \"3b844bd6-1b84-440b-935d-7881b340fa38\") " Sep 19 09:48:48.790044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.790006 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b844bd6-1b84-440b-935d-7881b340fa38-kube-api-access-jm49m" (OuterVolumeSpecName: "kube-api-access-jm49m") pod "3b844bd6-1b84-440b-935d-7881b340fa38" (UID: "3b844bd6-1b84-440b-935d-7881b340fa38"). InnerVolumeSpecName "kube-api-access-jm49m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:48:48.790044 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.790026 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3b844bd6-1b84-440b-935d-7881b340fa38" (UID: "3b844bd6-1b84-440b-935d-7881b340fa38"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:48:48.790199 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.790084 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3b844bd6-1b84-440b-935d-7881b340fa38" (UID: "3b844bd6-1b84-440b-935d-7881b340fa38"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:48:48.889454 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.889419 2585 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-webhook-cert\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:48:48.889454 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.889448 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm49m\" (UniqueName: \"kubernetes.io/projected/3b844bd6-1b84-440b-935d-7881b340fa38-kube-api-access-jm49m\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:48:48.889454 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:48.889458 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b844bd6-1b84-440b-935d-7881b340fa38-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:48:49.179172 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.179136 2585 generic.go:356] "Generic (PLEG): container finished" podID="3b844bd6-1b84-440b-935d-7881b340fa38" containerID="0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541" exitCode=0 Sep 19 09:48:49.179326 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.179180 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" Sep 19 09:48:49.179326 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.179191 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" event={"ID":"3b844bd6-1b84-440b-935d-7881b340fa38","Type":"ContainerDied","Data":"0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541"} Sep 19 09:48:49.179326 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.179238 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-c2xqr" event={"ID":"3b844bd6-1b84-440b-935d-7881b340fa38","Type":"ContainerDied","Data":"3519a987a4c80f46facb789eb4545245502a4977e50f62467cd48fbdfc92f395"} Sep 19 09:48:49.179326 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.179263 2585 scope.go:117] "RemoveContainer" containerID="0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541" Sep 19 09:48:49.188661 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.188641 2585 scope.go:117] "RemoveContainer" containerID="0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541" Sep 19 09:48:49.188937 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:48:49.188916 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541\": container with ID starting with 0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541 not found: ID does not exist" containerID="0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541" Sep 19 09:48:49.189008 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.188946 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541"} err="failed to get container status \"0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541\": rpc error: code = NotFound desc = could not find container \"0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541\": container with ID starting with 0bd407b8b1d78bb78f85db28ba61b51df2230dc22aa3cfd8f2db45748b1ea541 not found: ID does not exist" Sep 19 09:48:49.203314 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.203290 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-c2xqr"] Sep 19 09:48:49.210191 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.210167 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-c2xqr"] Sep 19 09:48:49.436741 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.436666 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b844bd6-1b84-440b-935d-7881b340fa38" path="/var/lib/kubelet/pods/3b844bd6-1b84-440b-935d-7881b340fa38/volumes" Sep 19 09:48:49.970322 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970276 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5fbf54c957-jmptt"] Sep 19 09:48:49.970733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970692 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b844bd6-1b84-440b-935d-7881b340fa38" containerName="manager" Sep 19 09:48:49.970733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970703 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b844bd6-1b84-440b-935d-7881b340fa38" containerName="manager" Sep 19 09:48:49.970733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970717 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" Sep 19 09:48:49.970733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970723 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8c5d2a3-1b5a-4684-a590-e02b08511345" containerName="istio-proxy" Sep 19 09:48:49.970733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970730 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" Sep 19 09:48:49.970733 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970736 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="908b0062-d7b4-424c-a70e-ab24750d0ad3" containerName="istio-proxy" Sep 19 09:48:49.970920 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.970802 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b844bd6-1b84-440b-935d-7881b340fa38" containerName="manager" Sep 19 09:48:49.975139 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.975122 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:49.978848 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.978829 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9txm5\"" Sep 19 09:48:49.978971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.978836 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 19 09:48:49.978971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.978883 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 19 09:48:49.979100 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.979034 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 19 09:48:49.984857 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.984835 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-jmptt"] Sep 19 09:48:49.998671 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.998637 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9004e5a-e90c-4ce6-82ab-33f5214f9214-metrics-certs\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:49.998808 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.998685 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9004e5a-e90c-4ce6-82ab-33f5214f9214-webhook-cert\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:49.998808 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:49.998737 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jzbv\" (UniqueName: \"kubernetes.io/projected/e9004e5a-e90c-4ce6-82ab-33f5214f9214-kube-api-access-4jzbv\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.099433 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.099384 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9004e5a-e90c-4ce6-82ab-33f5214f9214-webhook-cert\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.099644 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.099479 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jzbv\" (UniqueName: \"kubernetes.io/projected/e9004e5a-e90c-4ce6-82ab-33f5214f9214-kube-api-access-4jzbv\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.099644 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.099572 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9004e5a-e90c-4ce6-82ab-33f5214f9214-metrics-certs\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.101952 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.101922 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9004e5a-e90c-4ce6-82ab-33f5214f9214-webhook-cert\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.102079 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.101959 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9004e5a-e90c-4ce6-82ab-33f5214f9214-metrics-certs\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.116692 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.116659 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jzbv\" (UniqueName: \"kubernetes.io/projected/e9004e5a-e90c-4ce6-82ab-33f5214f9214-kube-api-access-4jzbv\") pod \"maas-controller-5fbf54c957-jmptt\" (UID: \"e9004e5a-e90c-4ce6-82ab-33f5214f9214\") " pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.286317 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.286230 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:50.566388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:50.566362 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fbf54c957-jmptt"] Sep 19 09:48:50.567429 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:48:50.567401 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9004e5a_e90c_4ce6_82ab_33f5214f9214.slice/crio-abbb1be6fb7c972f5a5fa9078a2fc282ecaf8f725c81e1f49e2b9f456f90a255 WatchSource:0}: Error finding container abbb1be6fb7c972f5a5fa9078a2fc282ecaf8f725c81e1f49e2b9f456f90a255: Status 404 returned error can't find the container with id abbb1be6fb7c972f5a5fa9078a2fc282ecaf8f725c81e1f49e2b9f456f90a255 Sep 19 09:48:51.193818 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:51.193725 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-jmptt" event={"ID":"e9004e5a-e90c-4ce6-82ab-33f5214f9214","Type":"ContainerStarted","Data":"2a6fa2ba643f73e91c1d6f2450728192800e44bba7c23f63afa463f43407cce9"} Sep 19 09:48:51.193818 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:51.193780 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fbf54c957-jmptt" event={"ID":"e9004e5a-e90c-4ce6-82ab-33f5214f9214","Type":"ContainerStarted","Data":"abbb1be6fb7c972f5a5fa9078a2fc282ecaf8f725c81e1f49e2b9f456f90a255"} Sep 19 09:48:51.194291 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:51.193925 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:48:51.217023 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:48:51.216966 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5fbf54c957-jmptt" podStartSLOduration=1.895940223 podStartE2EDuration="2.216940254s" totalImagesPullingTime="321.000031ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:48:51.215419469 +0000 UTC m=+2537.973540493" watchObservedRunningTime="2026-09-19 09:48:51.216940254 +0000 UTC m=+2537.975061281" Sep 19 09:49:02.201263 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:49:02.201228 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fbf54c957-jmptt" Sep 19 09:50:19.885134 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:19.885054 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv"] Sep 19 09:50:19.885625 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:19.885264 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" podUID="9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" containerName="manager" containerID="cri-o://5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306" gracePeriod=10 Sep 19 09:50:20.143870 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.143813 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:50:20.212025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.211996 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d967\" (UniqueName: \"kubernetes.io/projected/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-kube-api-access-4d967\") pod \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " Sep 19 09:50:20.212025 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.212029 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-extensions-socket-volume\") pod \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\" (UID: \"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3\") " Sep 19 09:50:20.212506 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.212476 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" (UID: "9a3cf710-71ea-4f2c-b201-3eb573a4a2a3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:50:20.214376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.214343 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-kube-api-access-4d967" (OuterVolumeSpecName: "kube-api-access-4d967") pod "9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" (UID: "9a3cf710-71ea-4f2c-b201-3eb573a4a2a3"). InnerVolumeSpecName "kube-api-access-4d967". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:50:20.312835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.312808 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d967\" (UniqueName: \"kubernetes.io/projected/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-kube-api-access-4d967\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:50:20.312835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.312832 2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3-extensions-socket-volume\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:50:20.667669 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.667638 2585 generic.go:356] "Generic (PLEG): container finished" podID="9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" containerID="5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306" exitCode=0 Sep 19 09:50:20.667793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.667677 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" Sep 19 09:50:20.667793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.667725 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" event={"ID":"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3","Type":"ContainerDied","Data":"5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306"} Sep 19 09:50:20.667793 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.667774 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv" event={"ID":"9a3cf710-71ea-4f2c-b201-3eb573a4a2a3","Type":"ContainerDied","Data":"6208d06f13f3335c489cb72be9f279a07b49b68a37eda593498efc4c5121b458"} Sep 19 09:50:20.667930 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.667796 2585 scope.go:117] "RemoveContainer" containerID="5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306" Sep 19 09:50:20.680381 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.680364 2585 scope.go:117] "RemoveContainer" containerID="5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306" Sep 19 09:50:20.680663 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:50:20.680643 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306\": container with ID starting with 5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306 not found: ID does not exist" containerID="5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306" Sep 19 09:50:20.680738 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.680670 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306"} err="failed to get container status \"5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306\": rpc error: code = NotFound desc = could not find container \"5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306\": container with ID starting with 5df8949c64aac3a71800c29ca61185ab52d4bcc78ffe367d6991199627015306 not found: ID does not exist" Sep 19 09:50:20.707407 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.707386 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv"] Sep 19 09:50:20.714028 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:20.714009 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w9qpv"] Sep 19 09:50:21.436130 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:50:21.436097 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" path="/var/lib/kubelet/pods/9a3cf710-71ea-4f2c-b201-3eb573a4a2a3/volumes" Sep 19 09:51:20.590218 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.590130 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8"] Sep 19 09:51:20.590811 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.590707 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" containerName="manager" Sep 19 09:51:20.590811 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.590727 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" containerName="manager" Sep 19 09:51:20.590932 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.590825 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a3cf710-71ea-4f2c-b201-3eb573a4a2a3" containerName="manager" Sep 19 09:51:20.593915 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.593897 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:20.598007 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.597984 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-x86cm\"" Sep 19 09:51:20.638680 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.638651 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxzsw\" (UniqueName: \"kubernetes.io/projected/372c3ca3-ef92-4e3e-9a99-f5f5904c7121-kube-api-access-kxzsw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8skc8\" (UID: \"372c3ca3-ef92-4e3e-9a99-f5f5904c7121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:20.638809 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.638698 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/372c3ca3-ef92-4e3e-9a99-f5f5904c7121-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8skc8\" (UID: \"372c3ca3-ef92-4e3e-9a99-f5f5904c7121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:20.646478 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.646443 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8"] Sep 19 09:51:20.739657 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.739621 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxzsw\" (UniqueName: \"kubernetes.io/projected/372c3ca3-ef92-4e3e-9a99-f5f5904c7121-kube-api-access-kxzsw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8skc8\" (UID: \"372c3ca3-ef92-4e3e-9a99-f5f5904c7121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:20.739839 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.739676 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/372c3ca3-ef92-4e3e-9a99-f5f5904c7121-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8skc8\" (UID: \"372c3ca3-ef92-4e3e-9a99-f5f5904c7121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:20.740071 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.740050 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/372c3ca3-ef92-4e3e-9a99-f5f5904c7121-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8skc8\" (UID: \"372c3ca3-ef92-4e3e-9a99-f5f5904c7121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:20.759340 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.759309 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxzsw\" (UniqueName: \"kubernetes.io/projected/372c3ca3-ef92-4e3e-9a99-f5f5904c7121-kube-api-access-kxzsw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8skc8\" (UID: \"372c3ca3-ef92-4e3e-9a99-f5f5904c7121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:20.904573 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:20.904471 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:21.207063 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:21.207036 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8"] Sep 19 09:51:21.208923 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:51:21.208886 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372c3ca3_ef92_4e3e_9a99_f5f5904c7121.slice/crio-9029b0740e2a3e6250cb337049de5a4ad6ea636dd419cf07673f89455bd20316 WatchSource:0}: Error finding container 9029b0740e2a3e6250cb337049de5a4ad6ea636dd419cf07673f89455bd20316: Status 404 returned error can't find the container with id 9029b0740e2a3e6250cb337049de5a4ad6ea636dd419cf07673f89455bd20316 Sep 19 09:51:21.997766 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:21.997731 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" event={"ID":"372c3ca3-ef92-4e3e-9a99-f5f5904c7121","Type":"ContainerStarted","Data":"b324a96fb5ac7258d91ac8d25c8e1f6542a0146142441ce55060f8b0f9dfb99d"} Sep 19 09:51:21.998149 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:21.997789 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:21.998149 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:21.997806 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" event={"ID":"372c3ca3-ef92-4e3e-9a99-f5f5904c7121","Type":"ContainerStarted","Data":"9029b0740e2a3e6250cb337049de5a4ad6ea636dd419cf07673f89455bd20316"} Sep 19 09:51:22.039724 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:22.039667 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" podStartSLOduration=2.039649964 podStartE2EDuration="2.039649964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:22.037632902 +0000 UTC m=+2688.795753930" watchObservedRunningTime="2026-09-19 09:51:22.039649964 +0000 UTC m=+2688.797770988" Sep 19 09:51:33.008361 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:33.008335 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8skc8" Sep 19 09:51:35.682642 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:35.682611 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:51:35.685868 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:35.685849 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:51:35.696379 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:35.696362 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:51:35.699514 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:51:35.699490 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6c442_4fe574be-a073-4960-803c-3d65227fc764/ovn-acl-logging/0.log" Sep 19 09:53:27.280554 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.280440 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67f76686b9-v5758"] Sep 19 09:53:27.284121 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.284095 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.293351 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.293321 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67f76686b9-v5758"] Sep 19 09:53:27.441315 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.441282 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b2ac916-be23-4ff7-9d2d-af19153e6579-metrics-certs\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.441497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.441326 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2ac916-be23-4ff7-9d2d-af19153e6579-maas-api-tls\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.441497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.441346 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2ac916-be23-4ff7-9d2d-af19153e6579-openshift-service-ca\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.441497 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.441400 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsrq9\" (UniqueName: \"kubernetes.io/projected/1b2ac916-be23-4ff7-9d2d-af19153e6579-kube-api-access-zsrq9\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.542587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.542480 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b2ac916-be23-4ff7-9d2d-af19153e6579-metrics-certs\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.542587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.542549 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2ac916-be23-4ff7-9d2d-af19153e6579-maas-api-tls\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.542827 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.542676 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2ac916-be23-4ff7-9d2d-af19153e6579-openshift-service-ca\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.542827 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.542748 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsrq9\" (UniqueName: \"kubernetes.io/projected/1b2ac916-be23-4ff7-9d2d-af19153e6579-kube-api-access-zsrq9\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.543394 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.543368 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b2ac916-be23-4ff7-9d2d-af19153e6579-openshift-service-ca\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.545393 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.545371 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b2ac916-be23-4ff7-9d2d-af19153e6579-metrics-certs\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.545494 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.545423 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b2ac916-be23-4ff7-9d2d-af19153e6579-maas-api-tls\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.550573 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.550544 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsrq9\" (UniqueName: \"kubernetes.io/projected/1b2ac916-be23-4ff7-9d2d-af19153e6579-kube-api-access-zsrq9\") pod \"maas-api-67f76686b9-v5758\" (UID: \"1b2ac916-be23-4ff7-9d2d-af19153e6579\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.596531 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.596489 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:27.873416 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.873388 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67f76686b9-v5758"] Sep 19 09:53:27.875669 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:53:27.875638 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2ac916_be23_4ff7_9d2d_af19153e6579.slice/crio-3f90caac81609b36baebe77bfdf35df009c75bfea5741b740f48b2fa4b4ca9d7 WatchSource:0}: Error finding container 3f90caac81609b36baebe77bfdf35df009c75bfea5741b740f48b2fa4b4ca9d7: Status 404 returned error can't find the container with id 3f90caac81609b36baebe77bfdf35df009c75bfea5741b740f48b2fa4b4ca9d7 Sep 19 09:53:27.877258 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:27.877224 2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:53:28.690427 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:28.690388 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" event={"ID":"1b2ac916-be23-4ff7-9d2d-af19153e6579","Type":"ContainerStarted","Data":"a186f6b95d3d4962b1ca705216e1e263c79f8ee4eedee2168fd54a4ad52a5fe8"} Sep 19 09:53:28.690934 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:28.690439 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" event={"ID":"1b2ac916-be23-4ff7-9d2d-af19153e6579","Type":"ContainerStarted","Data":"3f90caac81609b36baebe77bfdf35df009c75bfea5741b740f48b2fa4b4ca9d7"} Sep 19 09:53:28.690934 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:28.690469 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:28.708075 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:28.708027 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" podStartSLOduration=1.326345634 podStartE2EDuration="1.708011935s" totalImagesPullingTime="381.666301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:28.706383766 +0000 UTC m=+2815.464504792" watchObservedRunningTime="2026-09-19 09:53:28.708011935 +0000 UTC m=+2815.466132959" Sep 19 09:53:34.700589 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:34.700559 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67f76686b9-v5758" Sep 19 09:53:42.355835 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:42.355802 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-fqhvv"] Sep 19 09:53:42.356327 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:42.356002 2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" podUID="4c2372b3-d80a-4037-9743-9453b42e9299" containerName="maas-api" containerID="cri-o://5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a" gracePeriod=30 Sep 19 09:53:43.605709 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.605687 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:53:43.773723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.773635 2585 generic.go:356] "Generic (PLEG): container finished" podID="4c2372b3-d80a-4037-9743-9453b42e9299" containerID="5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a" exitCode=0 Sep 19 09:53:43.773723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.773681 2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" Sep 19 09:53:43.773723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.773696 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" event={"ID":"4c2372b3-d80a-4037-9743-9453b42e9299","Type":"ContainerDied","Data":"5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a"} Sep 19 09:53:43.773950 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.773746 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-fqhvv" event={"ID":"4c2372b3-d80a-4037-9743-9453b42e9299","Type":"ContainerDied","Data":"e7afbe608e85c96ab742c67104e0f53e7dd5fd647c1d0c2fe046857a9ef9515c"} Sep 19 09:53:43.773950 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.773775 2585 scope.go:117] "RemoveContainer" containerID="5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a" Sep 19 09:53:43.782388 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.782370 2585 scope.go:117] "RemoveContainer" containerID="5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a" Sep 19 09:53:43.782586 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.782568 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2372b3-d80a-4037-9743-9453b42e9299-openshift-service-ca\") pod \"4c2372b3-d80a-4037-9743-9453b42e9299\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " Sep 19 09:53:43.782674 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.782616 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-maas-api-tls\") pod \"4c2372b3-d80a-4037-9743-9453b42e9299\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " Sep 19 09:53:43.782721 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.782681 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hvq2\" (UniqueName: \"kubernetes.io/projected/4c2372b3-d80a-4037-9743-9453b42e9299-kube-api-access-6hvq2\") pod \"4c2372b3-d80a-4037-9743-9453b42e9299\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " Sep 19 09:53:43.782756 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.782729 2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs\") pod \"4c2372b3-d80a-4037-9743-9453b42e9299\" (UID: \"4c2372b3-d80a-4037-9743-9453b42e9299\") " Sep 19 09:53:43.782797 ip-10-0-132-57 kubenswrapper[2585]: E0919 09:53:43.782761 2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a\": container with ID starting with 5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a not found: ID does not exist" containerID="5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a" Sep 19 09:53:43.782852 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.782795 2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a"} err="failed to get container status \"5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a\": rpc error: code = NotFound desc = could not find container \"5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a\": container with ID starting with 5e6e69740284ee75e85d5def714d1f65c5ee1f54849838bada5b61e48849841a not found: ID does not exist" Sep 19 09:53:43.782956 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.782932 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2372b3-d80a-4037-9743-9453b42e9299-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c2372b3-d80a-4037-9743-9453b42e9299" (UID: "4c2372b3-d80a-4037-9743-9453b42e9299"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:43.784690 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.784660 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c2372b3-d80a-4037-9743-9453b42e9299" (UID: "4c2372b3-d80a-4037-9743-9453b42e9299"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:43.784778 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.784691 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c2372b3-d80a-4037-9743-9453b42e9299" (UID: "4c2372b3-d80a-4037-9743-9453b42e9299"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:43.784778 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.784762 2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2372b3-d80a-4037-9743-9453b42e9299-kube-api-access-6hvq2" (OuterVolumeSpecName: "kube-api-access-6hvq2") pod "4c2372b3-d80a-4037-9743-9453b42e9299" (UID: "4c2372b3-d80a-4037-9743-9453b42e9299"). InnerVolumeSpecName "kube-api-access-6hvq2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:53:43.883462 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.883440 2585 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-metrics-certs\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:53:43.883462 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.883463 2585 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2372b3-d80a-4037-9743-9453b42e9299-openshift-service-ca\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:53:43.883640 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.883472 2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c2372b3-d80a-4037-9743-9453b42e9299-maas-api-tls\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:53:43.883640 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:43.883480 2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hvq2\" (UniqueName: \"kubernetes.io/projected/4c2372b3-d80a-4037-9743-9453b42e9299-kube-api-access-6hvq2\") on node \"ip-10-0-132-57.ec2.internal\" DevicePath \"\"" Sep 19 09:53:44.101806 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:44.101775 2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-fqhvv"] Sep 19 09:53:44.105116 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:44.105086 2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-fqhvv"] Sep 19 09:53:45.436536 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:45.436493 2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2372b3-d80a-4037-9743-9453b42e9299" path="/var/lib/kubelet/pods/4c2372b3-d80a-4037-9743-9453b42e9299/volumes" Sep 19 09:53:56.979737 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:53:56.979706 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6479ff984f-b9qsf_51bf6be7-3c28-42ea-8153-2d53c67ef456/authorino/0.log" Sep 19 09:54:02.389282 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:02.389244 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9sqht_1336bfc7-f678-4c54-bea3-7ce8e78eda8e/manager/0.log" Sep 19 09:54:02.506875 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:02.506845 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bbk2z_ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b/manager/0.log" Sep 19 09:54:02.632847 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:02.632819 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fbf54c957-jmptt_e9004e5a-e90c-4ce6-82ab-33f5214f9214/manager/0.log" Sep 19 09:54:03.224379 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:03.224352 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-pkb62_9dfa6654-1703-433e-a300-07cfa060c67d/copy-manifests/0.log" Sep 19 09:54:03.243944 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:03.243916 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-pkb62_9dfa6654-1703-433e-a300-07cfa060c67d/manager/0.log" Sep 19 09:54:03.680046 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:03.680015 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-67f76686b9-v5758_1b2ac916-be23-4ff7-9d2d-af19153e6579/maas-api/0.log" Sep 19 09:54:03.997275 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:03.997197 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-pcnv9_780c2ea2-5d2a-4438-9dae-77b6e12dbe45/postgres/0.log" Sep 19 09:54:04.773390 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:04.773359 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6479ff984f-b9qsf_51bf6be7-3c28-42ea-8153-2d53c67ef456/authorino/0.log" Sep 19 09:54:04.891948 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:04.891913 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-b6wgm_61cd96ef-9cb3-47ec-b570-365e15622415/manager/0.log" Sep 19 09:54:05.003278 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:05.003244 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-wtv9r_14bebdf9-5b8e-4cb7-8467-9d45a6544e1b/manager/0.log" Sep 19 09:54:05.239728 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:05.239694 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-8skc8_372c3ca3-ef92-4e3e-9a99-f5f5904c7121/manager/0.log" Sep 19 09:54:05.474944 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:05.474911 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-5fsj5_1b78eae0-0e76-442b-996d-4219653d309e/manager/0.log" Sep 19 09:54:06.049354 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:06.049321 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-gj9ht_9688666a-3a15-4cc9-b226-4ab172e41fea/kube-auth-proxy/0.log" Sep 19 09:54:06.603843 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:06.603816 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56b8994dc8-sp2rh_f1c28be5-7f78-42b1-9126-fda3c48b9e04/router/0.log" Sep 19 09:54:06.946622 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:06.946539 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-g2q75_eedad42a-3003-4b50-b4e9-0137f683f144/main/0.log" Sep 19 09:54:07.171985 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:07.171952 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-f88ml_24058ab2-bda3-4d89-990e-549c45645c1f/main/0.log" Sep 19 09:54:12.788452 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.788418 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2bwqb/must-gather-wgh6v"] Sep 19 09:54:12.788882 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.788797 2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2372b3-d80a-4037-9743-9453b42e9299" containerName="maas-api" Sep 19 09:54:12.788882 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.788807 2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2372b3-d80a-4037-9743-9453b42e9299" containerName="maas-api" Sep 19 09:54:12.788882 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.788868 2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2372b3-d80a-4037-9743-9453b42e9299" containerName="maas-api" Sep 19 09:54:12.792128 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.792107 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:12.794976 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.794956 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2bwqb\"/\"openshift-service-ca.crt\"" Sep 19 09:54:12.795080 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.794956 2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2bwqb\"/\"default-dockercfg-5g8bj\"" Sep 19 09:54:12.795080 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.794990 2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2bwqb\"/\"kube-root-ca.crt\"" Sep 19 09:54:12.800971 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.800940 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bwqb/must-gather-wgh6v"] Sep 19 09:54:12.819283 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.819261 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fccf66dd-2c4a-4ee9-b6e7-c3665293085d-must-gather-output\") pod \"must-gather-wgh6v\" (UID: \"fccf66dd-2c4a-4ee9-b6e7-c3665293085d\") " pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:12.819372 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.819319 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfkwd\" (UniqueName: \"kubernetes.io/projected/fccf66dd-2c4a-4ee9-b6e7-c3665293085d-kube-api-access-wfkwd\") pod \"must-gather-wgh6v\" (UID: \"fccf66dd-2c4a-4ee9-b6e7-c3665293085d\") " pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:12.919787 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.919759 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfkwd\" (UniqueName: \"kubernetes.io/projected/fccf66dd-2c4a-4ee9-b6e7-c3665293085d-kube-api-access-wfkwd\") pod \"must-gather-wgh6v\" (UID: \"fccf66dd-2c4a-4ee9-b6e7-c3665293085d\") " pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:12.919911 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.919837 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fccf66dd-2c4a-4ee9-b6e7-c3665293085d-must-gather-output\") pod \"must-gather-wgh6v\" (UID: \"fccf66dd-2c4a-4ee9-b6e7-c3665293085d\") " pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:12.920115 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.920098 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fccf66dd-2c4a-4ee9-b6e7-c3665293085d-must-gather-output\") pod \"must-gather-wgh6v\" (UID: \"fccf66dd-2c4a-4ee9-b6e7-c3665293085d\") " pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:12.927706 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:12.927687 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfkwd\" (UniqueName: \"kubernetes.io/projected/fccf66dd-2c4a-4ee9-b6e7-c3665293085d-kube-api-access-wfkwd\") pod \"must-gather-wgh6v\" (UID: \"fccf66dd-2c4a-4ee9-b6e7-c3665293085d\") " pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:13.102105 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:13.102080 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bwqb/must-gather-wgh6v" Sep 19 09:54:13.372958 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:13.372926 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bwqb/must-gather-wgh6v"] Sep 19 09:54:13.374453 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:54:13.374427 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfccf66dd_2c4a_4ee9_b6e7_c3665293085d.slice/crio-b68b536005da39e5c47991d8eb7fa873799b8f09eaeda9fcc8dd57f91aa93524 WatchSource:0}: Error finding container b68b536005da39e5c47991d8eb7fa873799b8f09eaeda9fcc8dd57f91aa93524: Status 404 returned error can't find the container with id b68b536005da39e5c47991d8eb7fa873799b8f09eaeda9fcc8dd57f91aa93524 Sep 19 09:54:13.933849 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:13.933810 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bwqb/must-gather-wgh6v" event={"ID":"fccf66dd-2c4a-4ee9-b6e7-c3665293085d","Type":"ContainerStarted","Data":"b68b536005da39e5c47991d8eb7fa873799b8f09eaeda9fcc8dd57f91aa93524"} Sep 19 09:54:14.945337 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:14.945303 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bwqb/must-gather-wgh6v" event={"ID":"fccf66dd-2c4a-4ee9-b6e7-c3665293085d","Type":"ContainerStarted","Data":"ad0e335f7d0fa04762a00ed9815f61d125ed6ba365f696924ee86fb87cff650f"} Sep 19 09:54:14.945792 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:14.945351 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bwqb/must-gather-wgh6v" event={"ID":"fccf66dd-2c4a-4ee9-b6e7-c3665293085d","Type":"ContainerStarted","Data":"dc7503cc43bb2f4290972f593752cf4d5fabf3d2a18f00f3df888aa4bcfe4cd8"} Sep 19 09:54:14.962216 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:14.962169 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2bwqb/must-gather-wgh6v" podStartSLOduration=2.270746011 podStartE2EDuration="2.962153997s" totalImagesPullingTime="691.407986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:54:14.96055208 +0000 UTC m=+2861.718673107" watchObservedRunningTime="2026-09-19 09:54:14.962153997 +0000 UTC m=+2861.720275022" Sep 19 09:54:15.567769 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:15.567739 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fljvj_6f74b8f0-71dd-482c-aa43-b65acd2cb16b/global-pull-secret-syncer/0.log" Sep 19 09:54:15.802777 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:15.802740 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lrw5b_6661651c-f4e1-4a3b-9b68-310a36c28f6b/konnectivity-agent/0.log" Sep 19 09:54:15.894416 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:15.894329 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-57.ec2.internal_03b66c5394879f7702479f341fee142c/haproxy/0.log" Sep 19 09:54:20.190199 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:20.189431 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6479ff984f-b9qsf_51bf6be7-3c28-42ea-8153-2d53c67ef456/authorino/0.log" Sep 19 09:54:20.252499 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:20.252471 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-b6wgm_61cd96ef-9cb3-47ec-b570-365e15622415/manager/0.log" Sep 19 09:54:20.289580 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:20.288857 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-wtv9r_14bebdf9-5b8e-4cb7-8467-9d45a6544e1b/manager/0.log" Sep 19 09:54:20.382314 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:20.381899 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-8skc8_372c3ca3-ef92-4e3e-9a99-f5f5904c7121/manager/0.log" Sep 19 09:54:20.482541 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:20.482326 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-5fsj5_1b78eae0-0e76-442b-996d-4219653d309e/manager/0.log" Sep 19 09:54:22.471491 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:22.471461 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g8r9_96cfb1a6-633f-4db0-b70b-762ab4ef9447/node-exporter/0.log" Sep 19 09:54:22.496508 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:22.496475 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g8r9_96cfb1a6-633f-4db0-b70b-762ab4ef9447/kube-rbac-proxy/0.log" Sep 19 09:54:22.518723 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:22.518698 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7g8r9_96cfb1a6-633f-4db0-b70b-762ab4ef9447/init-textfile/0.log" Sep 19 09:54:23.047438 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:23.047405 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-mzl8k_33bbb5a8-1c66-474f-a849-ed184d180eef/prometheus-operator-admission-webhook/0.log" Sep 19 09:54:25.390942 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.390911 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/1.log" Sep 19 09:54:25.401594 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.401564 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-kx6t7_fb7d6fe0-eeee-4a99-a9be-c83ee47ccd26/console-operator/2.log" Sep 19 09:54:25.888539 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.888482 2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c"] Sep 19 09:54:25.895255 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.895224 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:25.914316 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.914279 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c"] Sep 19 09:54:25.977145 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.977099 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-podres\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:25.977334 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.977158 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-lib-modules\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:25.977334 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.977273 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-sys\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:25.977423 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.977363 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-proc\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:25.977476 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:25.977458 2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qxn5\" (UniqueName: \"kubernetes.io/projected/25ceabbb-a0c0-422d-8170-711267dbfcfa-kube-api-access-4qxn5\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.046935 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.046903 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dc5b6d78b-6fl6p_068ce7f8-360f-483d-a256-77eeca4f631c/console/0.log" Sep 19 09:54:26.078367 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078330 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-proc\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078408 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-proc\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078423 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qxn5\" (UniqueName: \"kubernetes.io/projected/25ceabbb-a0c0-422d-8170-711267dbfcfa-kube-api-access-4qxn5\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078468 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-podres\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078510 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-lib-modules\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078587 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078563 2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-sys\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078654 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-podres\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078663 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-sys\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.078826 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.078705 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/25ceabbb-a0c0-422d-8170-711267dbfcfa-lib-modules\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.087774 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.087745 2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qxn5\" (UniqueName: \"kubernetes.io/projected/25ceabbb-a0c0-422d-8170-711267dbfcfa-kube-api-access-4qxn5\") pod \"perf-node-gather-daemonset-htq9c\" (UID: \"25ceabbb-a0c0-422d-8170-711267dbfcfa\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.101486 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.101465 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-f5l9p_55c3e3b3-a08d-4145-bc84-31733b991710/download-server/0.log" Sep 19 09:54:26.208968 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.208871 2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:26.527845 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:26.527805 2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c"] Sep 19 09:54:26.529296 ip-10-0-132-57 kubenswrapper[2585]: W0919 09:54:26.529267 2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod25ceabbb_a0c0_422d_8170_711267dbfcfa.slice/crio-68c9f6213578101fca322c5212dbd3d4c893086b0a3267e0097c105db3f0cd78 WatchSource:0}: Error finding container 68c9f6213578101fca322c5212dbd3d4c893086b0a3267e0097c105db3f0cd78: Status 404 returned error can't find the container with id 68c9f6213578101fca322c5212dbd3d4c893086b0a3267e0097c105db3f0cd78 Sep 19 09:54:27.059081 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:27.059035 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" event={"ID":"25ceabbb-a0c0-422d-8170-711267dbfcfa","Type":"ContainerStarted","Data":"078657dba61087a9902e8f9660c50943d731f60cad7a3db360bcac208af8b599"} Sep 19 09:54:27.059297 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:27.059107 2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" event={"ID":"25ceabbb-a0c0-422d-8170-711267dbfcfa","Type":"ContainerStarted","Data":"68c9f6213578101fca322c5212dbd3d4c893086b0a3267e0097c105db3f0cd78"} Sep 19 09:54:27.059996 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:27.059969 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:27.078809 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:27.078741 2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" podStartSLOduration=2.07871399 podStartE2EDuration="2.07871399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:54:27.075158639 +0000 UTC m=+2873.833279667" watchObservedRunningTime="2026-09-19 09:54:27.07871399 +0000 UTC m=+2873.836835016" Sep 19 09:54:28.005203 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:28.005170 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvr64_7cf31f40-1983-41a0-8bdd-88249a000156/dns/0.log" Sep 19 09:54:28.026823 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:28.026794 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvr64_7cf31f40-1983-41a0-8bdd-88249a000156/kube-rbac-proxy/0.log" Sep 19 09:54:28.097539 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:28.097485 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wcqst_2f96adac-d4cf-4684-b8cf-2a87800cbf10/dns-node-resolver/0.log" Sep 19 09:54:28.659747 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:28.659717 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b7768f645-vgq4t_33361761-7682-4d9f-8cba-8954d27deb57/registry/0.log" Sep 19 09:54:28.703901 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:28.703871 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kghq6_c1e4bdae-16ac-478a-be9d-b108434edc8b/node-ca/0.log" Sep 19 09:54:30.091500 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:30.091470 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-gj9ht_9688666a-3a15-4cc9-b226-4ab172e41fea/kube-auth-proxy/0.log" Sep 19 09:54:30.351102 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:30.350998 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56b8994dc8-sp2rh_f1c28be5-7f78-42b1-9126-fda3c48b9e04/router/0.log" Sep 19 09:54:31.224905 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:31.224873 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-48xg4_c40b15ed-37e4-476a-a915-ae88b2ef8e37/serve-healthcheck-canary/0.log" Sep 19 09:54:31.779958 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:31.779929 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wb9f_26f43266-21f8-4171-9548-09b85d106b6f/kube-rbac-proxy/0.log" Sep 19 09:54:31.800530 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:31.800488 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wb9f_26f43266-21f8-4171-9548-09b85d106b6f/exporter/0.log" Sep 19 09:54:31.822269 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:31.822238 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2wb9f_26f43266-21f8-4171-9548-09b85d106b6f/extractor/0.log" Sep 19 09:54:34.040376 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:34.040341 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9sqht_1336bfc7-f678-4c54-bea3-7ce8e78eda8e/manager/0.log" Sep 19 09:54:34.088671 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:34.088638 2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-htq9c" Sep 19 09:54:34.479643 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:34.479610 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bbk2z_ca760d1f-c1ed-4bbf-8ffc-cf3b1461719b/manager/0.log" Sep 19 09:54:34.556018 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:34.555933 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fbf54c957-jmptt_e9004e5a-e90c-4ce6-82ab-33f5214f9214/manager/0.log" Sep 19 09:54:34.868907 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:34.868863 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-pkb62_9dfa6654-1703-433e-a300-07cfa060c67d/manager/0.log" Sep 19 09:54:34.885403 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:34.885378 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-pkb62_9dfa6654-1703-433e-a300-07cfa060c67d/copy-manifests/0.log" Sep 19 09:54:36.161695 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:36.161618 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-gj9ht_9688666a-3a15-4cc9-b226-4ab172e41fea/kube-auth-proxy/0.log" Sep 19 09:54:36.236387 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:36.236350 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zm5zz_ac0fba72-bc18-4119-b8ab-8ad6d7c62b9d/openshift-lws-operator/0.log" Sep 19 09:54:36.254544 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:36.254454 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56b8994dc8-sp2rh_f1c28be5-7f78-42b1-9126-fda3c48b9e04/router/0.log" Sep 19 09:54:38.250143 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:38.250111 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-gj9ht_9688666a-3a15-4cc9-b226-4ab172e41fea/kube-auth-proxy/0.log" Sep 19 09:54:38.370144 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:38.370036 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56b8994dc8-sp2rh_f1c28be5-7f78-42b1-9126-fda3c48b9e04/router/0.log" Sep 19 09:54:42.729256 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.729225 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5jch7_01b00b26-83e4-49bd-bc69-04501dadedd4/kube-multus/0.log" Sep 19 09:54:42.749924 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.749882 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtr4_76ea5338-fc9a-45a1-9111-a727e59130fa/kube-multus-additional-cni-plugins/0.log" Sep 19 09:54:42.768400 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.768364 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtr4_76ea5338-fc9a-45a1-9111-a727e59130fa/egress-router-binary-copy/0.log" Sep 19 09:54:42.787094 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.787075 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtr4_76ea5338-fc9a-45a1-9111-a727e59130fa/cni-plugins/0.log" Sep 19 09:54:42.805456 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.805432 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtr4_76ea5338-fc9a-45a1-9111-a727e59130fa/bond-cni-plugin/0.log" Sep 19 09:54:42.824464 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.824441 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtr4_76ea5338-fc9a-45a1-9111-a727e59130fa/routeoverride-cni/0.log" Sep 19 09:54:42.843697 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.843670 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtr4_76ea5338-fc9a-45a1-9111-a727e59130fa/whereabouts-cni-bincopy/0.log" Sep 19 09:54:42.862064 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:42.862031 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtr4_76ea5338-fc9a-45a1-9111-a727e59130fa/whereabouts-cni/0.log" Sep 19 09:54:43.311539 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:43.311485 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nst5s_3f5f3742-e891-4188-aa1d-c48c5e8e8a0b/network-metrics-daemon/0.log" Sep 19 09:54:43.328266 ip-10-0-132-57 kubenswrapper[2585]: I0919 09:54:43.328243 2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nst5s_3f5f3742-e891-4188-aa1d-c48c5e8e8a0b/kube-rbac-proxy/0.log"